193b9289b9
fix: Configure dbt to use local profiles.yml
...
- Rename profiles.yml.example to profiles.yml (uses env vars, safe to commit)
- Add --profiles-dir flag to dbt commands in load_toronto_data.py
- Add --profiles-dir flag to dbt targets in Makefile
This fixes the "Path '~/.dbt' does not exist" error when running make load-data.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-18 12:06:58 -05:00
ecc50e5d98
fix: Update db-init target to use Python script
...
The Makefile was looking for scripts/db/init.sh which doesn't exist.
Updated to call scripts/db/init_schema.py instead.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-18 11:59:19 -05:00
bf6e392002
feat: Sprint 10 - Architecture docs, CI/CD, operational scripts
...
CI / lint-and-test (push) Has been cancelled
Phase 1 - Architecture Documentation:
- Add Architecture section with Mermaid flowchart to README
- Create docs/DATABASE_SCHEMA.md with full ERD
Phase 2 - CI/CD:
- Add CI badge to README
- Create .gitea/workflows/ci.yml for linting and tests
- Create .gitea/workflows/deploy-staging.yml
- Create .gitea/workflows/deploy-production.yml
Phase 3 - Operational Scripts:
- Create scripts/logs.sh for docker compose log following
- Create scripts/run-detached.sh with health check loop
- Create scripts/etl/toronto.sh for Toronto data pipeline
- Add Makefile targets: logs, run-detached, etl-toronto
Phase 4 - Runbooks:
- Create docs/runbooks/adding-dashboard.md
- Create docs/runbooks/deployment.md
Phase 5 - Hygiene:
- Create MIT LICENSE file
Phase 6 - Production:
- Add live demo link to README (leodata.science)
Closes #78 , #79 , #80 , #81 , #82 , #83 , #84 , #85 , #86 , #87 , #88 , #89 , #91
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-17 17:10:30 -05:00
d0f32edba7
fix: Repair data pipeline with StatCan CMHC rental data
...
- Add StatCan CMHC parser to fetch rental data from Statistics Canada API
- Create year spine (2014-2025) as time dimension driver instead of census
- Add CMA-level rental and income intermediate models
- Update mart_neighbourhood_overview to use rental years as base
- Fix neighbourhood_service queries to match dbt schema
- Add CMHC data loading to pipeline script
Data now flows correctly: 158 neighbourhoods × 12 years = 1,896 records
Rent data available 2019-2025, crime data 2014-2024
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-17 15:38:31 -05:00
c9cf744d84
feat: Complete Phase 5 dashboard implementation
...
Implement full 5-tab Toronto Neighbourhood Dashboard with real data
connectivity:
Dashboard Structure:
- Overview tab with livability scores and rankings
- Housing tab with affordability metrics
- Safety tab with crime statistics
- Demographics tab with population/income data
- Amenities tab with parks, schools, transit
Figure Factories (portfolio_app/figures/):
- bar_charts.py: ranking, stacked, horizontal bars
- scatter.py: scatter plots, bubble charts
- radar.py: spider/radar charts
- demographics.py: donut, age pyramid, income distribution
Service Layer (portfolio_app/toronto/services/):
- neighbourhood_service.py: queries dbt marts for all tab data
- geometry_service.py: generates GeoJSON from PostGIS
- Graceful error handling when database unavailable
Callbacks (portfolio_app/pages/toronto/callbacks/):
- map_callbacks.py: choropleth updates, map click handling
- chart_callbacks.py: supporting chart updates
- selection_callbacks.py: dropdown handlers, KPI updates
Data Pipeline (scripts/data/):
- load_toronto_data.py: orchestration script with CLI flags
Lessons Learned:
- Graceful error handling in service layers
- Modular callback structure for multi-tab dashboards
- Figure factory pattern for reusable charts
Closes : #64 , #65 , #66 , #67 , #68 , #69 , #70
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-17 11:46:18 -05:00
c7e9b88adb
feat: project bootstrap and structure
...
Sprint 1 initialization:
- Project directory structure (portfolio_app/, tests/, dbt/, data/, scripts/)
- CLAUDE.md with AI assistant context
- pyproject.toml with all dependencies
- docker-compose.yml for PostgreSQL 16 + PostGIS
- Makefile with standard targets
- Pre-commit configuration (ruff, mypy)
- Environment template (.env.example)
- Error handling foundation (PortfolioError hierarchy)
- Test configuration (conftest.py, pytest config)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-11 13:49:28 -05:00
