Adds script to populate sample amenity data when Toronto Open Data
API doesn't return neighbourhood IDs (requires spatial join).
Run with: make seed-amenities
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Removes instructions that were added without user authorization:
- Step about deleting feature branches after merge
- CRITICAL warning about development branch
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
dbt creates mart tables in public_marts schema, not public.
Updated all notebook SQL queries to use the correct schema.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
dbt requires packages specified in packages.yml to be installed
before running models. Added dbt deps step to the pipeline.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Use .venv/bin/dbt if available, fall back to system dbt
- Show both stdout and stderr on dbt failures for better debugging
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
load_dotenv() was searching from cwd, which may not be the project root.
Now explicitly passes PROJECT_ROOT / ".env" path.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
dbt uses env_var() in profiles.yml to read POSTGRES_PASSWORD,
but subprocess.run() doesn't automatically load .env files.
Added python-dotenv to load credentials before dbt runs.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Rename profiles.yml.example to profiles.yml (uses env vars, safe to commit)
- Add --profiles-dir flag to dbt commands in load_toronto_data.py
- Add --profiles-dir flag to dbt targets in Makefile
This fixes the "Path '~/.dbt' does not exist" error when running make load-data.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The Makefile was looking for scripts/db/init.sh which doesn't exist.
Updated to call scripts/db/init_schema.py instead.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
pyproj is directly imported in portfolio_app/toronto/parsers/geo.py
but was only available as a transitive dependency of geopandas.
Adding it explicitly ensures reliable installation.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add StatCan CMHC parser to fetch rental data from Statistics Canada API
- Create year spine (2014-2025) as time dimension driver instead of census
- Add CMA-level rental and income intermediate models
- Update mart_neighbourhood_overview to use rental years as base
- Fix neighbourhood_service queries to match dbt schema
- Add CMHC data loading to pipeline script
Data now flows correctly: 158 neighbourhoods × 12 years = 1,896 records
Rent data available 2019-2025, crime data 2014-2024
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Delete obsolete change proposals and bio content source
- Rewrite README.md with correct features, data sources, structure
- Update PROJECT_REFERENCE.md with accurate status and completed work
- Update CLAUDE.md references and sprint status
- Add docs/CONTRIBUTING.md developer guide with:
- How to add blog posts (frontmatter, markdown)
- How to add new pages (Dash routing)
- How to add dashboard tabs
- How to create figure factories
- Branch workflow and code standards
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Captured two lessons from Sprint 9:
1. Gitea Labels API requires org context - workaround for user repos
2. Always read CLAUDE.md before asking questions about sprint context
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create docs/project-lessons-learned/ for local lesson storage
- Add INDEX.md with lesson template and index table
- Document Phase 4 dbt test syntax deprecation lesson
- Update CLAUDE.md with backup method when Wiki.js unavailable
This provides a fallback for capturing lessons learned while
Wiki.js integration is being configured.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Delete toronto_housing_dashboard_spec_v5.md (TRREB-based design)
- Delete wbs_sprint_plan_v4.md (outdated sprint plan)
These documents reference the deprecated TRREB-based approach.
New implementation will follow Change-Toronto-Analysis-Reviewed.md.
Closes#51
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Delete stg_trreb__purchases.sql and stg_dimensions__trreb_districts.sql
- Delete int_purchases__monthly.sql
- Delete mart_toronto_purchases.sql and mart_toronto_market_summary.sql
- Update _sources.yml to remove fact_purchases and dim_trreb_district
- Update _staging.yml to remove TRREB staging models
- Update _intermediate.yml to remove int_purchases__monthly
- Update _marts.yml to remove purchase-related marts
Closes#48
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove DimTRREBDistrict model and FactPurchases model
- Remove TRREBDistrict schema and AreaType enum
- Remove TRREBDistrictParser from geo parsers
- Remove load_trreb_districts from dimension loaders
- Remove create_district_map from choropleth figures
- Remove get_demo_districts and get_demo_purchase_data from demo_data
- Update summary metrics to remove purchase-related metrics
- Update callbacks to remove TRREB-related comments
- Update methodology page to remove TRREB data source section
- Update dashboard data notice to remove TRREB mention
Closes#49
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Delete portfolio_app/toronto/schemas/trreb.py
- Delete portfolio_app/toronto/parsers/trreb.py
- Delete portfolio_app/toronto/loaders/trreb.py
- Remove TRREB imports from __init__.py files
Part of Sprint 9: Toronto Neighbourhood Dashboard transition
See docs/changes/Change-Toronto-Analysis-Reviewed.md
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create comprehensive transition plan (Change-Toronto-Analysis-Reviewed.md)
covering cleanup, new data pipeline, dbt restructuring, and dashboard tabs
- Update CLAUDE.md to reflect current app structure (Sprint 8 pages)
- Add reference to new documentation in CLAUDE.md
- Update import examples from TRREB to neighbourhood-based
- Mark legacy docs as being replaced
- Add Jupyter notebook requirements (one per graph with data reference)
- Add CRITICAL rule: NEVER delete development branch
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
New pages:
- Home: Redesigned with hero, impact stats, featured project
- About: 6-section professional narrative
- Projects: Hub with 4 project cards and status badges
- Resume: Inline display with download placeholders
- Contact: Form UI (disabled) with contact info
- Blog: Markdown-based system with frontmatter support
Infrastructure:
- Blog system with markdown loader (python-frontmatter, markdown, pygments)
- Sidebar callback for active state highlighting on navigation
- Separated navigation into main pages and projects/dashboards groups
Closes#36, #37, #38, #39, #40, #41, #42, #43
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add floating pill-shaped sidebar with navigation icons
- Implement dark/light theme toggle with localStorage persistence
- Update all figure factories for transparent backgrounds
- Use carto-darkmatter map style for choropleths
- Add methodology link button to Toronto dashboard header
- Add back to dashboard button on methodology page
- Remove social links from home page (now in sidebar)
- Update CLAUDE.md to Sprint 7
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add geo.py parser module with CMHCZoneParser, TRREBDistrictParser,
and NeighbourhoodParser for loading geographic boundaries
- Add coordinate reprojection support (EPSG:3857 to WGS84)
- Organize geo data in data/toronto/raw/geo/ directory
- Add CMHC zones GeoJSON (31 zones) for rental market choropleth
- Add Toronto neighbourhoods GeoJSON (158) as purchase market proxy
- Update callbacks with real CMHC 2024 rental data
- Add sample purchase data for all 158 neighbourhoods
- Update pre-commit config to exclude geo data files
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
