feat: Sprint 6 polish - methodology, demo data, deployment prep
- Add policy event markers to time series charts - Create methodology page (/toronto/methodology) with data sources - Add demo data module for testing without full pipeline - Update README with project documentation - Add health check endpoint (/health) - Add database initialization script - Export new figure factory functions Closes #21 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
120
README.md
120
README.md
@@ -1,2 +1,120 @@
|
||||
# personal-portfolio
|
||||
# Analytics Portfolio
|
||||
|
||||
A data analytics portfolio showcasing end-to-end data engineering, visualization, and analysis capabilities.
|
||||
|
||||
## Projects
|
||||
|
||||
### Toronto Housing Dashboard
|
||||
|
||||
An interactive choropleth dashboard analyzing Toronto's housing market using multi-source data integration.
|
||||
|
||||
**Features:**
|
||||
- Purchase market analysis from TRREB monthly reports
|
||||
- Rental market analysis from CMHC annual surveys
|
||||
- Interactive choropleth maps by district/zone
|
||||
- Time series visualization with policy event annotations
|
||||
- Purchase/Rental mode toggle
|
||||
|
||||
**Data Sources:**
|
||||
- [TRREB Market Watch](https://trreb.ca/market-data/market-watch/) - Monthly purchase statistics
|
||||
- [CMHC Rental Market Survey](https://www.cmhc-schl.gc.ca/professionals/housing-markets-data-and-research/housing-data/data-tables/rental-market) - Annual rental data
|
||||
|
||||
**Tech Stack:**
|
||||
- Python 3.11+ / Dash / Plotly
|
||||
- PostgreSQL + PostGIS
|
||||
- dbt for data transformation
|
||||
- Pydantic for validation
|
||||
- SQLAlchemy 2.0
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Clone and setup
|
||||
git clone https://github.com/lmiranda/personal-portfolio.git
|
||||
cd personal-portfolio
|
||||
|
||||
# Install dependencies and configure environment
|
||||
make setup
|
||||
|
||||
# Start database
|
||||
make docker-up
|
||||
|
||||
# Initialize database schema
|
||||
make db-init
|
||||
|
||||
# Run development server
|
||||
make run
|
||||
```
|
||||
|
||||
Visit `http://localhost:8050` to view the portfolio.
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
portfolio_app/
|
||||
├── app.py # Dash app factory
|
||||
├── config.py # Pydantic settings
|
||||
├── pages/
|
||||
│ ├── home.py # Bio landing page (/)
|
||||
│ └── toronto/ # Toronto dashboard (/toronto)
|
||||
├── components/ # Shared UI components
|
||||
├── figures/ # Plotly figure factories
|
||||
└── toronto/ # Toronto data logic
|
||||
├── parsers/ # PDF/CSV extraction
|
||||
├── loaders/ # Database operations
|
||||
├── schemas/ # Pydantic models
|
||||
└── models/ # SQLAlchemy ORM
|
||||
|
||||
dbt/
|
||||
├── models/
|
||||
│ ├── staging/ # 1:1 source tables
|
||||
│ ├── intermediate/ # Business logic
|
||||
│ └── marts/ # Analytical tables
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
make test # Run tests
|
||||
make lint # Run linter
|
||||
make format # Format code
|
||||
make ci # Run all checks
|
||||
```
|
||||
|
||||
## Data Pipeline
|
||||
|
||||
```
|
||||
Raw Files (PDF/Excel)
|
||||
↓
|
||||
Parsers (pdfplumber, pandas)
|
||||
↓
|
||||
Pydantic Validation
|
||||
↓
|
||||
SQLAlchemy Loaders
|
||||
↓
|
||||
PostgreSQL + PostGIS
|
||||
↓
|
||||
dbt Transformations
|
||||
↓
|
||||
Dash Visualization
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Copy `.env.example` to `.env` and configure:
|
||||
|
||||
```bash
|
||||
DATABASE_URL=postgresql://user:pass@localhost:5432/portfolio
|
||||
POSTGRES_USER=portfolio
|
||||
POSTGRES_PASSWORD=<secure>
|
||||
POSTGRES_DB=portfolio
|
||||
DASH_DEBUG=true
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
|
||||
## Author
|
||||
|
||||
Leo Miranda - [GitHub](https://github.com/lmiranda) | [LinkedIn](https://linkedin.com/in/yourprofile)
|
||||
|
||||
Reference in New Issue
Block a user