lmiranda
140d3085bf
- Add policy event markers to time series charts - Create methodology page (/toronto/methodology) with data sources - Add demo data module for testing without full pipeline - Update README with project documentation - Add health check endpoint (/health) - Add database initialization script - Export new figure factory functions Closes #21 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Analytics Portfolio
A data analytics portfolio showcasing end-to-end data engineering, visualization, and analysis capabilities.
Projects
Toronto Housing Dashboard
An interactive choropleth dashboard analyzing Toronto's housing market using multi-source data integration.
Features:
- Purchase market analysis from TRREB monthly reports
- Rental market analysis from CMHC annual surveys
- Interactive choropleth maps by district/zone
- Time series visualization with policy event annotations
- Purchase/Rental mode toggle
Data Sources:
- TRREB Market Watch - Monthly purchase statistics
- CMHC Rental Market Survey - Annual rental data
Tech Stack:
- Python 3.11+ / Dash / Plotly
- PostgreSQL + PostGIS
- dbt for data transformation
- Pydantic for validation
- SQLAlchemy 2.0
Quick Start
# Clone and setup
git clone https://github.com/lmiranda/personal-portfolio.git
cd personal-portfolio
# Install dependencies and configure environment
make setup
# Start database
make docker-up
# Initialize database schema
make db-init
# Run development server
make run
Visit http://localhost:8050 to view the portfolio.
Project Structure
portfolio_app/
├── app.py # Dash app factory
├── config.py # Pydantic settings
├── pages/
│ ├── home.py # Bio landing page (/)
│ └── toronto/ # Toronto dashboard (/toronto)
├── components/ # Shared UI components
├── figures/ # Plotly figure factories
└── toronto/ # Toronto data logic
├── parsers/ # PDF/CSV extraction
├── loaders/ # Database operations
├── schemas/ # Pydantic models
└── models/ # SQLAlchemy ORM
dbt/
├── models/
│ ├── staging/ # 1:1 source tables
│ ├── intermediate/ # Business logic
│ └── marts/ # Analytical tables
Development
make test # Run tests
make lint # Run linter
make format # Format code
make ci # Run all checks
Data Pipeline
Raw Files (PDF/Excel)
↓
Parsers (pdfplumber, pandas)
↓
Pydantic Validation
↓
SQLAlchemy Loaders
↓
PostgreSQL + PostGIS
↓
dbt Transformations
↓
Dash Visualization
Environment Variables
Copy .env.example to .env and configure:
DATABASE_URL=postgresql://user:pass@localhost:5432/portfolio
POSTGRES_USER=portfolio
POSTGRES_PASSWORD=<secure>
POSTGRES_DB=portfolio
DASH_DEBUG=true
License
MIT