lmiranda
140d3085bf
- Add policy event markers to time series charts - Create methodology page (/toronto/methodology) with data sources - Add demo data module for testing without full pipeline - Update README with project documentation - Add health check endpoint (/health) - Add database initialization script - Export new figure factory functions Closes #21 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
121 lines
2.8 KiB
Markdown
121 lines
2.8 KiB
Markdown
# Analytics Portfolio
|
|
|
|
A data analytics portfolio showcasing end-to-end data engineering, visualization, and analysis capabilities.
|
|
|
|
## Projects
|
|
|
|
### Toronto Housing Dashboard
|
|
|
|
An interactive choropleth dashboard analyzing Toronto's housing market using multi-source data integration.
|
|
|
|
**Features:**
|
|
- Purchase market analysis from TRREB monthly reports
|
|
- Rental market analysis from CMHC annual surveys
|
|
- Interactive choropleth maps by district/zone
|
|
- Time series visualization with policy event annotations
|
|
- Purchase/Rental mode toggle
|
|
|
|
**Data Sources:**
|
|
- [TRREB Market Watch](https://trreb.ca/market-data/market-watch/) - Monthly purchase statistics
|
|
- [CMHC Rental Market Survey](https://www.cmhc-schl.gc.ca/professionals/housing-markets-data-and-research/housing-data/data-tables/rental-market) - Annual rental data
|
|
|
|
**Tech Stack:**
|
|
- Python 3.11+ / Dash / Plotly
|
|
- PostgreSQL + PostGIS
|
|
- dbt for data transformation
|
|
- Pydantic for validation
|
|
- SQLAlchemy 2.0
|
|
|
|
## Quick Start
|
|
|
|
```bash
|
|
# Clone and setup
|
|
git clone https://github.com/lmiranda/personal-portfolio.git
|
|
cd personal-portfolio
|
|
|
|
# Install dependencies and configure environment
|
|
make setup
|
|
|
|
# Start database
|
|
make docker-up
|
|
|
|
# Initialize database schema
|
|
make db-init
|
|
|
|
# Run development server
|
|
make run
|
|
```
|
|
|
|
Visit `http://localhost:8050` to view the portfolio.
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
portfolio_app/
|
|
├── app.py # Dash app factory
|
|
├── config.py # Pydantic settings
|
|
├── pages/
|
|
│ ├── home.py # Bio landing page (/)
|
|
│ └── toronto/ # Toronto dashboard (/toronto)
|
|
├── components/ # Shared UI components
|
|
├── figures/ # Plotly figure factories
|
|
└── toronto/ # Toronto data logic
|
|
├── parsers/ # PDF/CSV extraction
|
|
├── loaders/ # Database operations
|
|
├── schemas/ # Pydantic models
|
|
└── models/ # SQLAlchemy ORM
|
|
|
|
dbt/
|
|
├── models/
|
|
│ ├── staging/ # 1:1 source tables
|
|
│ ├── intermediate/ # Business logic
|
|
│ └── marts/ # Analytical tables
|
|
```
|
|
|
|
## Development
|
|
|
|
```bash
|
|
make test # Run tests
|
|
make lint # Run linter
|
|
make format # Format code
|
|
make ci # Run all checks
|
|
```
|
|
|
|
## Data Pipeline
|
|
|
|
```
|
|
Raw Files (PDF/Excel)
|
|
↓
|
|
Parsers (pdfplumber, pandas)
|
|
↓
|
|
Pydantic Validation
|
|
↓
|
|
SQLAlchemy Loaders
|
|
↓
|
|
PostgreSQL + PostGIS
|
|
↓
|
|
dbt Transformations
|
|
↓
|
|
Dash Visualization
|
|
```
|
|
|
|
## Environment Variables
|
|
|
|
Copy `.env.example` to `.env` and configure:
|
|
|
|
```bash
|
|
DATABASE_URL=postgresql://user:pass@localhost:5432/portfolio
|
|
POSTGRES_USER=portfolio
|
|
POSTGRES_PASSWORD=<secure>
|
|
POSTGRES_DB=portfolio
|
|
DASH_DEBUG=true
|
|
```
|
|
|
|
## License
|
|
|
|
MIT
|
|
|
|
## Author
|
|
|
|
Leo Miranda - [GitHub](https://github.com/lmiranda) | [LinkedIn](https://linkedin.com/in/yourprofile)
|