120 lines
4.2 KiB
Markdown
120 lines
4.2 KiB
Markdown
# Lesson Learned #003: Requirements File Organization
|
|
|
|
## Issue Name
|
|
Missing main requirements.txt for local development and testing
|
|
|
|
## Description
|
|
After implementing comprehensive test coverage, tests could not be run locally due to missing dependencies. The project had separate `requirements-backend.txt` and `requirements-frontend.txt` files for Docker containers, but no unified requirements file for local development.
|
|
|
|
## Error Messages
|
|
```
|
|
ModuleNotFoundError: No module named 'structlog'
|
|
ModuleNotFoundError: No module named 'pytest'
|
|
```
|
|
|
|
## Root Cause
|
|
1. **Fragmented Dependencies**: Backend and frontend requirements were split into separate files for Docker optimization
|
|
2. **Missing Local Setup**: No unified requirements file for local development and testing
|
|
3. **Documentation Gap**: README didn't clearly explain how to install dependencies for local testing
|
|
|
|
## Solution Implemented
|
|
|
|
### 1. Created Main Requirements File
|
|
- **File**: `requirements.txt`
|
|
- **Purpose**: Combined all dependencies for local development
|
|
- **Content**: Merged backend and frontend requirements
|
|
|
|
### 2. Created Development Requirements File
|
|
- **File**: `dev-requirements.txt`
|
|
- **Purpose**: Testing and development dependencies only
|
|
- **Content**: pytest, black, flake8, mypy, and core dependencies needed for testing
|
|
|
|
### 3. Updated Documentation
|
|
- **File**: `README.md`
|
|
- **Section**: Quick Start
|
|
- **Addition**: Local development setup instructions with proper pip install commands
|
|
|
|
### 4. Maintained Docker Optimization
|
|
- **Approach**: Kept separate `requirements-backend.txt` and `requirements-frontend.txt` for Docker containers
|
|
- **Benefit**: Smaller container images with only necessary dependencies
|
|
|
|
## File Structure Created
|
|
```
|
|
job-forge/
|
|
├── requirements.txt # All dependencies for local development
|
|
├── dev-requirements.txt # Development and testing dependencies only
|
|
├── requirements-backend.txt # Backend container dependencies (existing)
|
|
├── requirements-frontend.txt # Frontend container dependencies (existing)
|
|
└── README.md # Updated with local setup instructions
|
|
```
|
|
|
|
## Prevention Strategy
|
|
|
|
### 1. Requirements File Standards
|
|
- **Main Requirements**: Always maintain a unified `requirements.txt` for local development
|
|
- **Development Requirements**: Separate `dev-requirements.txt` for testing tools
|
|
- **Container Requirements**: Keep optimized files for Docker containers
|
|
|
|
### 2. Documentation Requirements
|
|
- **Installation Instructions**: Clear pip install commands in README
|
|
- **Testing Setup**: Document how to run tests locally vs in containers
|
|
- **Dependencies Explanation**: Explain the purpose of each requirements file
|
|
|
|
### 3. Testing Integration
|
|
- **Local Testing**: Ensure tests can run with local pip-installed dependencies
|
|
- **Container Testing**: Maintain ability to test within Docker environment
|
|
- **CI/CD Integration**: Use appropriate requirements file for each environment
|
|
|
|
## Implementation Details
|
|
|
|
### Requirements.txt Content
|
|
```
|
|
# Combined requirements for local development
|
|
fastapi==0.109.2
|
|
uvicorn[standard]==0.27.1
|
|
# ... (all backend and frontend dependencies)
|
|
pytest==8.0.2
|
|
pytest-asyncio==0.23.5
|
|
# ... (all testing dependencies)
|
|
```
|
|
|
|
### Dev-Requirements.txt Content
|
|
```
|
|
# Development and testing only
|
|
pytest==8.0.2
|
|
pytest-asyncio==0.23.5
|
|
pytest-cov==4.0.0
|
|
black==24.2.0
|
|
# ... (minimal set for testing)
|
|
```
|
|
|
|
### README Update
|
|
```bash
|
|
# For local development and testing
|
|
pip install -r requirements.txt
|
|
|
|
# For development dependencies only
|
|
pip install -r dev-requirements.txt
|
|
|
|
# Run tests locally
|
|
python validate_tests.py
|
|
python run_tests.py
|
|
pytest
|
|
```
|
|
|
|
## Key Takeaways
|
|
|
|
1. **Multiple Requirements Files**: Different environments need different dependency sets
|
|
2. **Local Development Priority**: Always provide easy local setup for developers
|
|
3. **Documentation Clarity**: Clear installation instructions prevent frustration
|
|
4. **Container Optimization**: Keep container-specific requirements minimal and focused
|
|
|
|
## Status
|
|
✅ **RESOLVED** - Created unified requirements files and updated documentation
|
|
|
|
## Related Files
|
|
- `requirements.txt` (new)
|
|
- `dev-requirements.txt` (new)
|
|
- `README.md` (updated)
|
|
- `requirements-backend.txt` (existing, unchanged)
|
|
- `requirements-frontend.txt` (existing, unchanged) |