Major documentation overhaul: Transform to Python/FastAPI web application

This comprehensive update transforms Job Forge from a generic MVP concept to a production-ready Python/FastAPI web application prototype with complete documentation, testing infrastructure, and deployment procedures. ## 🏗️ Architecture Changes - Updated all documentation to reflect Python/FastAPI + Dash + PostgreSQL stack - Transformed from MVP concept to deployable web application prototype - Added comprehensive multi-tenant architecture with Row Level Security (RLS) - Integrated Claude API and OpenAI API for AI-powered document generation ## 📚 Documentation Overhaul - **CLAUDE.md**: Complete rewrite as project orchestrator for 4 specialized agents - **README.md**: New centralized documentation hub with organized navigation - **API Specification**: Updated with comprehensive FastAPI endpoint documentation - **Database Design**: Enhanced schema with RLS policies and performance optimization - **Architecture Guide**: Transformed to web application focus with deployment strategy ## 🏗️ New Documentation Structure - **docs/development/**: Python/FastAPI coding standards and development guidelines - **docs/infrastructure/**: Docker setup and server deployment procedures - **docs/testing/**: Comprehensive QA procedures with pytest integration - **docs/ai/**: AI prompt templates and examples (preserved from original) ## 🎯 Team Structure Updates - **.claude/agents/**: 4 new Python/FastAPI specialized agents - simplified_technical_lead.md: Architecture and technical guidance - fullstack_developer.md: FastAPI backend + Dash frontend implementation - simplified_qa.md: pytest testing and quality assurance - simplified_devops.md: Docker deployment and server infrastructure ## 🧪 Testing Infrastructure - **pytest.ini**: Complete pytest configuration with coverage requirements - **tests/conftest.py**: Comprehensive test fixtures and database setup - **tests/unit/**: Example unit tests for auth and application services - **tests/integration/**: API integration test examples - Support for async testing, AI service mocking, and database testing ## 🧹 Cleanup - Removed 9 duplicate/outdated documentation files - Eliminated conflicting technology references (Node.js/TypeScript) - Consolidated overlapping content into comprehensive guides - Cleaned up project structure for professional development workflow ## 🚀 Production Ready Features - Docker containerization for development and production - Server deployment procedures for prototype hosting - Security best practices with JWT authentication and RLS - Performance optimization with database indexing and caching - Comprehensive testing strategy with quality gates This update establishes Job Forge as a professional Python/FastAPI web application prototype ready for development and deployment. 🤖 Generated with Claude Code (https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-02 11:33:32 -04:00
parent d9a8b13c16
commit b646e2f5df
41 changed files with 10237 additions and 5499 deletions
--- a/docs/infrastructure/server_deployment.md
+++ b/docs/infrastructure/server_deployment.md
@@ -0,0 +1,530 @@
+# Server Deployment Guide - Job Forge
+
+## Overview
+
+This guide covers deploying Job Forge to your own server for prototype development and testing. The deployment uses Docker containers for easy management and isolation.
+
+## Prerequisites
+
+### Server Requirements
+- **OS**: Ubuntu 20.04+ or CentOS 8+
+- **RAM**: Minimum 2GB, recommended 4GB
+- **Storage**: Minimum 20GB available disk space
+- **CPU**: 2 cores recommended
+- **Network**: Public IP address and domain name (optional)
+
+### Required Software
+- Docker 20.10+
+- Docker Compose 2.0+
+- Git
+- Nginx (for reverse proxy)
+- Certbot (for SSL certificates)
+
+## Initial Server Setup
+
+### 1. Update System
+```bash
+# Ubuntu/Debian
+sudo apt update && sudo apt upgrade -y
+
+# CentOS/RHEL
+sudo yum update -y
+```
+
+### 2. Install Docker
+```bash
+# Ubuntu/Debian
+curl -fsSL https://get.docker.com -o get-docker.sh
+sudo sh get-docker.sh
+sudo usermod -aG docker $USER
+
+# CentOS/RHEL
+sudo yum install -y docker
+sudo systemctl start docker
+sudo systemctl enable docker
+sudo usermod -aG docker $USER
+```
+
+### 3. Install Docker Compose
+```bash
+sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
+sudo chmod +x /usr/local/bin/docker-compose
+```
+
+### 4. Install Additional Tools
+```bash
+# Ubuntu/Debian
+sudo apt install -y git nginx certbot python3-certbot-nginx
+
+# CentOS/RHEL
+sudo yum install -y git nginx certbot python3-certbot-nginx
+```
+
+## Application Deployment
+
+### 1. Clone Repository
+```bash
+cd /opt
+sudo git clone https://github.com/yourusername/job-forge.git
+sudo chown -R $USER:$USER job-forge
+cd job-forge
+```
+
+### 2. Environment Configuration
+```bash
+# Copy environment template
+cp .env.example .env
+
+# Edit environment variables
+nano .env
+```
+
+#### Required Environment Variables
+```bash
+# Database Configuration
+DATABASE_URL="postgresql://jobforge:CHANGE_THIS_PASSWORD@postgres:5432/jobforge"
+DATABASE_POOL_SIZE=10
+
+# AI Service API Keys (obtain from providers)
+CLAUDE_API_KEY="your-claude-api-key-here"
+OPENAI_API_KEY="your-openai-api-key-here"
+
+# Security Settings
+JWT_SECRET="your-super-secret-jwt-key-minimum-32-characters"
+JWT_ALGORITHM="HS256"
+JWT_EXPIRE_MINUTES=1440
+
+# Application Settings
+APP_NAME="Job Forge"
+DEBUG=false
+LOG_LEVEL="INFO"
+
+# Server Configuration
+SERVER_HOST="0.0.0.0"
+SERVER_PORT=8000
+WORKERS=2
+
+# Security
+ALLOWED_HOSTS=["yourdomain.com", "www.yourdomain.com", "localhost"]
+CORS_ORIGINS=["https://yourdomain.com", "https://www.yourdomain.com"]
+```
+
+### 3. Create Required Directories
+```bash
+mkdir -p uploads backups logs ssl
+chmod 755 uploads backups logs
+chmod 700 ssl
+```
+
+### 4. Build and Start Services
+```bash
+# Build Docker images
+docker-compose build
+
+# Start services in background
+docker-compose up -d
+
+# Check status
+docker-compose ps
+```
+
+### 5. Initialize Database
+```bash
+# Run database migrations
+docker-compose exec jobforge-app alembic upgrade head
+
+# Verify database setup
+docker-compose exec postgres psql -U jobforge -d jobforge -c "\dt"
+```
+
+### 6. Verify Application
+```bash
+# Check application health
+curl http://localhost:8000/health
+
+# Check logs
+docker-compose logs jobforge-app
+```
+
+## Nginx Configuration
+
+### 1. Create Nginx Configuration
+```bash
+sudo nano /etc/nginx/sites-available/jobforge
+```
+
+```nginx
+server {
+    listen 80;
+    server_name yourdomain.com www.yourdomain.com;
+    
+    # Redirect to HTTPS
+    return 301 https://$server_name$request_uri;
+}
+
+server {
+    listen 443 ssl http2;
+    server_name yourdomain.com www.yourdomain.com;
+
+    # SSL Configuration (will be added by certbot)
+    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;
+    
+    # Security headers
+    add_header X-Frame-Options "SAMEORIGIN" always;
+    add_header X-XSS-Protection "1; mode=block" always;
+    add_header X-Content-Type-Options "nosniff" always;
+    add_header Referrer-Policy "no-referrer-when-downgrade" always;
+    add_header Content-Security-Policy "default-src 'self' http: https: data: blob: 'unsafe-inline'" always;
+
+    # File upload size limit
+    client_max_body_size 10M;
+
+    # Main application
+    location / {
+        proxy_pass http://localhost:8000;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+        proxy_redirect off;
+        
+        # Timeout settings for AI operations
+        proxy_connect_timeout 60s;
+        proxy_send_timeout 60s;
+        proxy_read_timeout 120s;
+    }
+
+    # Health check endpoint
+    location /health {
+        proxy_pass http://localhost:8000/health;
+        access_log off;
+    }
+
+    # Static files (if any)
+    location /static/ {
+        alias /opt/job-forge/static/;
+        expires 30d;
+        add_header Cache-Control "public, immutable";
+    }
+
+    # API documentation
+    location /docs {
+        proxy_pass http://localhost:8000/docs;
+    }
+
+    # Deny access to sensitive files
+    location ~ /\. {
+        deny all;
+    }
+}
+```
+
+### 2. Enable Site and Test Configuration
+```bash
+# Enable site
+sudo ln -s /etc/nginx/sites-available/jobforge /etc/nginx/sites-enabled/
+
+# Test configuration
+sudo nginx -t
+
+# Restart nginx
+sudo systemctl restart nginx
+```
+
+## SSL Certificate Setup
+
+### 1. Obtain SSL Certificate
+```bash
+# Using Let's Encrypt Certbot
+sudo certbot --nginx -d yourdomain.com -d www.yourdomain.com
+
+# Follow prompts to configure SSL
+```
+
+### 2. Configure Auto-Renewal
+```bash
+# Test renewal
+sudo certbot renew --dry-run
+
+# Add cron job for auto-renewal
+sudo crontab -e
+
+# Add this line:
+0 12 * * * /usr/bin/certbot renew --quiet
+```
+
+## Firewall Configuration
+
+### 1. Configure UFW (Ubuntu)
+```bash
+sudo ufw allow ssh
+sudo ufw allow 'Nginx Full'
+sudo ufw --force enable
+sudo ufw status
+```
+
+### 2. Configure Firewalld (CentOS)
+```bash
+sudo firewall-cmd --permanent --add-service=ssh
+sudo firewall-cmd --permanent --add-service=http
+sudo firewall-cmd --permanent --add-service=https
+sudo firewall-cmd --reload
+```
+
+## Monitoring and Maintenance
+
+### 1. Log Management
+```bash
+# View application logs
+docker-compose logs -f jobforge-app
+
+# View nginx logs
+sudo tail -f /var/log/nginx/access.log
+sudo tail -f /var/log/nginx/error.log
+
+# Set up log rotation
+sudo nano /etc/logrotate.d/jobforge
+```
+
+```
+/opt/job-forge/logs/*.log {
+    daily
+    missingok
+    rotate 30
+    compress
+    delaycompress
+    notifempty
+    copytruncate
+}
+```
+
+### 2. Backup Configuration
+```bash
+# Create backup script
+sudo nano /opt/job-forge/backup.sh
+```
+
+```bash
+#!/bin/bash
+# Job Forge backup script
+
+BACKUP_DIR="/opt/backups/jobforge"
+DATE=$(date +%Y%m%d_%H%M%S)
+
+# Create backup directory
+mkdir -p "$BACKUP_DIR"
+
+echo "Starting Job Forge backup - $DATE"
+
+# Database backup
+docker-compose exec -T postgres pg_dump -U jobforge jobforge | gzip > "$BACKUP_DIR/database_$DATE.sql.gz"
+
+# Application files backup
+tar -czf "$BACKUP_DIR/app_$DATE.tar.gz" \
+    --exclude="logs/*" \
+    --exclude="uploads/*" \
+    /opt/job-forge
+
+# Keep only last 30 days
+find "$BACKUP_DIR" -name "*.gz" -mtime +30 -delete
+
+echo "Backup completed successfully"
+```
+
+```bash
+# Make executable
+chmod +x /opt/job-forge/backup.sh
+
+# Add to crontab (daily backup at 2 AM)
+sudo crontab -e
+0 2 * * * /opt/job-forge/backup.sh
+```
+
+### 3. Health Monitoring
+```bash
+# Create health check script
+nano /opt/job-forge/health-check.sh
+```
+
+```bash
+#!/bin/bash
+# Job Forge health check
+
+HEALTH_URL="http://localhost:8000/health"
+LOG_FILE="/opt/job-forge/logs/health-check.log"
+
+response=$(curl -s -o /dev/null -w "%{http_code}" "$HEALTH_URL")
+
+if [ "$response" = "200" ]; then
+    echo "$(date): Health check passed" >> "$LOG_FILE"
+else
+    echo "$(date): Health check failed - HTTP $response" >> "$LOG_FILE"
+    # Send alert (email, Slack, etc.)
+    # systemctl restart docker-compose@job-forge
+fi
+```
+
+```bash
+# Make executable and add to cron (every 5 minutes)
+chmod +x /opt/job-forge/health-check.sh
+crontab -e
+*/5 * * * * /opt/job-forge/health-check.sh
+```
+
+## Application Updates
+
+### 1. Update Process
+```bash
+cd /opt/job-forge
+
+# Create backup before update
+./backup.sh
+
+# Pull latest changes
+git pull origin main
+
+# Rebuild and restart services
+docker-compose build
+docker-compose up -d
+
+# Run database migrations if needed
+docker-compose exec jobforge-app alembic upgrade head
+
+# Verify application is working
+curl http://localhost:8000/health
+```
+
+### 2. Rollback Process
+```bash
+# If update fails, rollback to previous version
+git log --oneline -10  # Find previous commit
+git checkout <previous-commit-hash>
+
+# Rebuild and restart
+docker-compose build
+docker-compose up -d
+
+# Or restore from backup if needed
+```
+
+## Troubleshooting
+
+### Common Issues
+
+#### 1. Application Won't Start
+```bash
+# Check logs
+docker-compose logs jobforge-app
+
+# Check database connection
+docker-compose exec postgres pg_isready -U jobforge
+
+# Check environment variables
+docker-compose exec jobforge-app env | grep -E "(DATABASE|CLAUDE|OPENAI)"
+```
+
+#### 2. Database Connection Issues
+```bash
+# Restart postgres
+docker-compose restart postgres
+
+# Check database logs
+docker-compose logs postgres
+
+# Connect to database manually
+docker-compose exec postgres psql -U jobforge -d jobforge
+```
+
+#### 3. SSL Certificate Issues
+```bash
+# Check certificate status
+sudo certbot certificates
+
+# Renew certificate manually
+sudo certbot renew
+
+# Check nginx configuration
+sudo nginx -t
+```
+
+#### 4. Permission Issues
+```bash
+# Fix file permissions
+sudo chown -R $USER:$USER /opt/job-forge
+chmod 755 /opt/job-forge/uploads
+chmod 700 /opt/job-forge/ssl
+```
+
+### Performance Optimization
+
+#### 1. Database Optimization
+```bash
+# Connect to database
+docker-compose exec postgres psql -U jobforge -d jobforge
+
+# Check slow queries
+SELECT query, mean_time, calls FROM pg_stat_statements ORDER BY mean_time DESC LIMIT 10;
+
+# Analyze table statistics
+ANALYZE;
+```
+
+#### 2. Container Resource Limits
+```yaml
+# In docker-compose.yml, add resource limits
+services:
+  jobforge-app:
+    deploy:
+      resources:
+        limits:
+          cpus: '1.0'
+          memory: 1G
+        reservations:
+          cpus: '0.5'
+          memory: 512M
+```
+
+#### 3. Nginx Caching
+```nginx
+# Add to nginx configuration
+location ~* \.(jpg|jpeg|png|gif|ico|css|js)$ {
+    expires 1y;
+    add_header Cache-Control "public, immutable";
+}
+```
+
+## Security Hardening
+
+### 1. System Updates
+```bash
+# Enable automatic security updates
+sudo apt install unattended-upgrades
+sudo dpkg-reconfigure -plow unattended-upgrades
+```
+
+### 2. Fail2Ban Setup
+```bash
+# Install fail2ban
+sudo apt install fail2ban
+
+# Configure for nginx
+sudo nano /etc/fail2ban/jail.local
+```
+
+```ini
+[nginx-http-auth]
+enabled = true
+
+[nginx-limit-req]
+enabled = true
+```
+
+### 3. Docker Security
+```bash
+# Run containers as non-root user (already configured in Dockerfile)
+# Limit container capabilities
+# Use secrets for sensitive data
+```
+
+This deployment guide provides a comprehensive setup for Job Forge on your own server. Adjust configurations based on your specific requirements and security policies.