Commit Graph

14 Commits

Author SHA1 Message Date
cda2a078d9 refactor(dbt): migrate to domain-scoped schema names
Some checks failed
CI / lint-and-test (pull_request) Has been cancelled
- Create generate_schema_name macro to use custom schema names directly
- Update dbt_project.yml schemas: staging→stg_toronto, intermediate→int_toronto, marts→mart_toronto
- Add dbt/macros/toronto/ directory for future domain-specific macros
- Fix documentation drift in PROJECT_REFERENCE.md (load-data-only→load-toronto-only)
- Update DATABASE_SCHEMA.md with new schema names
- Update CLAUDE.md database schemas table
- Update adding-dashboard.md runbook with domain-scoped pattern

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 12:32:39 -05:00
62d1a52eed refactor: multi-dashboard structural migration
Some checks failed
CI / lint-and-test (pull_request) Has been cancelled
- Rename dbt project from toronto_housing to portfolio
- Restructure dbt models into domain subdirectories:
  - shared/ for cross-domain dimensions (dim_time)
  - staging/toronto/, intermediate/toronto/, marts/toronto/
- Update SQLAlchemy models for raw_toronto schema
- Add explicit cross-schema FK relationships for FactRentals
- Namespace figure factories under figures/toronto/
- Namespace notebooks under notebooks/toronto/
- Update Makefile with domain-specific targets and env loading
- Update all documentation for multi-dashboard structure

This enables adding new dashboard projects (e.g., /football, /energy)
without structural conflicts or naming collisions.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 19:08:20 -05:00
263b52d5e4 docs: sync documentation with codebase
Some checks failed
CI / lint-and-test (push) Has been cancelled
Fixes identified by doc-guardian audit:

Critical fixes:
- DATABASE_SCHEMA.md: Fix staging model name stg_police__crimes → stg_toronto__crime
- DATABASE_SCHEMA.md: Update mart model names to match actual dbt models
- CLAUDE.md: Fix errors/ description (no handlers module exists)
- scripts/etl/toronto.sh: Fix parser module references to actual modules

Stale fixes:
- CONTRIBUTING.md: Add make typecheck, test-cov; fix make ci description
- PROJECT_REFERENCE.md: Document services/, callback modules, all Makefile targets
- CLAUDE.md: Expand Makefile commands, add plugin documentation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 16:25:29 -05:00
54665bac63 revert: Remove unauthorized branch workflow instructions
Some checks failed
CI / lint-and-test (push) Has been cancelled
Removes instructions that were added without user authorization:
- Step about deleting feature branches after merge
- CRITICAL warning about development branch

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-18 20:36:09 -05:00
d0f32edba7 fix: Repair data pipeline with StatCan CMHC rental data
- Add StatCan CMHC parser to fetch rental data from Statistics Canada API
- Create year spine (2014-2025) as time dimension driver instead of census
- Add CMA-level rental and income intermediate models
- Update mart_neighbourhood_overview to use rental years as base
- Fix neighbourhood_service queries to match dbt schema
- Add CMHC data loading to pipeline script

Data now flows correctly: 158 neighbourhoods × 12 years = 1,896 records
Rent data available 2019-2025, crime data 2014-2024

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-17 15:38:31 -05:00
4818c53fd2 docs: Rewrite documentation with accurate project state
- Delete obsolete change proposals and bio content source
- Rewrite README.md with correct features, data sources, structure
- Update PROJECT_REFERENCE.md with accurate status and completed work
- Update CLAUDE.md references and sprint status
- Add docs/CONTRIBUTING.md developer guide with:
  - How to add blog posts (frontmatter, markdown)
  - How to add new pages (Dash routing)
  - How to add dashboard tabs
  - How to create figure factories
  - Branch workflow and code standards

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-17 12:27:25 -05:00
1eba95d4d1 docs: Complete Phase 6 notebooks and Phase 7 documentation review
Phase 6 - Jupyter Notebooks (15 total):
- Overview tab: livability_choropleth, top_bottom_10_bar, income_safety_scatter
- Housing tab: affordability_choropleth, rent_trend_line, tenure_breakdown_bar
- Safety tab: crime_rate_choropleth, crime_breakdown_bar, crime_trend_line
- Demographics tab: income_choropleth, age_distribution, population_density_bar
- Amenities tab: amenity_index_choropleth, amenity_radar, transit_accessibility_bar

Phase 7 - Documentation:
- Updated CLAUDE.md with Sprint 9 completion status
- Added notebooks directory to application structure
- Expanded figures directory listing

Closes #71, #72, #73, #74, #75, #76, #77

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-17 12:10:46 -05:00
3054441630 docs: Add local lessons learned backup system
- Create docs/project-lessons-learned/ for local lesson storage
- Add INDEX.md with lesson template and index table
- Document Phase 4 dbt test syntax deprecation lesson
- Update CLAUDE.md with backup method when Wiki.js unavailable

This provides a fallback for capturing lessons learned while
Wiki.js integration is being configured.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 11:52:06 -05:00
053acf6436 feat: Implement Phase 3 neighbourhood data model
Add schemas, parsers, loaders, and models for Toronto neighbourhood-centric
data including census profiles, crime statistics, and amenities.

Schemas:
- NeighbourhoodRecord, CensusRecord, CrimeRecord, CrimeType
- AmenityType, AmenityRecord, AmenityCount

Models:
- BridgeCMHCNeighbourhood (zone-to-neighbourhood mapping with weights)
- FactCensus, FactCrime, FactAmenities

Parsers:
- TorontoOpenDataParser (CKAN API for neighbourhoods, census, amenities)
- TorontoPoliceParser (crime rates, MCI data)

Loaders:
- load_census_data, load_crime_data, load_amenities
- build_cmhc_neighbourhood_crosswalk (PostGIS area weights)

Also updates CLAUDE.md with projman plugin workflow documentation.

Closes #53, #54, #55, #56, #57, #58, #59

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 11:07:13 -05:00
81993b23a7 docs: Update CLAUDE.md and PROJECT_REFERENCE.md for neighbourhood transition
CLAUDE.md:
- Update project status to Sprint 9
- Remove TRREB references from data model section
- Update star schema to reflect current tables
- Simplify deferred features section
- Update reference documents

PROJECT_REFERENCE.md:
- Update import examples to neighbourhood-based
- Update data sources for neighbourhood dashboard
- Update geographic reality diagram
- Update star schema
- Modernize sprint overview
- Update scope boundaries
- Update success criteria with completed milestones
- Update reference documents

Closes #52

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 10:17:22 -05:00
d3ca4ad4eb docs: Add implementation plan for neighbourhood dashboard transition
- Create comprehensive transition plan (Change-Toronto-Analysis-Reviewed.md)
  covering cleanup, new data pipeline, dbt restructuring, and dashboard tabs
- Update CLAUDE.md to reflect current app structure (Sprint 8 pages)
- Add reference to new documentation in CLAUDE.md
- Update import examples from TRREB to neighbourhood-based
- Mark legacy docs as being replaced
- Add Jupyter notebook requirements (one per graph with data reference)
- Add CRITICAL rule: NEVER delete development branch

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 09:29:57 -05:00
b3fb94c7cb feat: Add floating sidebar navigation and dark theme support
- Add floating pill-shaped sidebar with navigation icons
- Implement dark/light theme toggle with localStorage persistence
- Update all figure factories for transparent backgrounds
- Use carto-darkmatter map style for choropleths
- Add methodology link button to Toronto dashboard header
- Add back to dashboard button on methodology page
- Remove social links from home page (now in sidebar)
- Update CLAUDE.md to Sprint 7

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-15 11:53:13 -05:00
10f46f7cf1 docs: update CLAUDE.md reference doc paths
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 14:02:39 -05:00
c7e9b88adb feat: project bootstrap and structure
Sprint 1 initialization:
- Project directory structure (portfolio_app/, tests/, dbt/, data/, scripts/)
- CLAUDE.md with AI assistant context
- pyproject.toml with all dependencies
- docker-compose.yml for PostgreSQL 16 + PostGIS
- Makefile with standard targets
- Pre-commit configuration (ruff, mypy)
- Environment template (.env.example)
- Error handling foundation (PortfolioError hierarchy)
- Test configuration (conftest.py, pytest config)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 13:49:28 -05:00