Commit Graph

52 Commits

Author SHA1 Message Date
b6d210ec6b feat: Implement Phase 4 dbt model restructuring
Create neighbourhood-centric dbt transformation layer:

Staging (5 models):
- stg_toronto__neighbourhoods - Neighbourhood dimension
- stg_toronto__census - Census demographics
- stg_toronto__crime - Crime statistics
- stg_toronto__amenities - Amenity counts
- stg_cmhc__zone_crosswalk - Zone-to-neighbourhood weights

Intermediate (5 models):
- int_neighbourhood__demographics - Combined census with quintiles
- int_neighbourhood__housing - Housing + affordability indicators
- int_neighbourhood__crime_summary - Aggregated crime with YoY
- int_neighbourhood__amenity_scores - Per-capita amenity metrics
- int_rentals__neighbourhood_allocated - CMHC via area weights

Marts (5 models):
- mart_neighbourhood_overview - Composite livability score
- mart_neighbourhood_housing - Affordability index
- mart_neighbourhood_safety - Crime rates per 100K
- mart_neighbourhood_demographics - Income/age indices
- mart_neighbourhood_amenities - Amenity index

Closes #60, #61, #62, #63

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 11:41:27 -05:00
053acf6436 feat: Implement Phase 3 neighbourhood data model
Add schemas, parsers, loaders, and models for Toronto neighbourhood-centric
data including census profiles, crime statistics, and amenities.

Schemas:
- NeighbourhoodRecord, CensusRecord, CrimeRecord, CrimeType
- AmenityType, AmenityRecord, AmenityCount

Models:
- BridgeCMHCNeighbourhood (zone-to-neighbourhood mapping with weights)
- FactCensus, FactCrime, FactAmenities

Parsers:
- TorontoOpenDataParser (CKAN API for neighbourhoods, census, amenities)
- TorontoPoliceParser (crime rates, MCI data)

Loaders:
- load_census_data, load_crime_data, load_amenities
- build_cmhc_neighbourhood_crosswalk (PostGIS area weights)

Also updates CLAUDE.md with projman plugin workflow documentation.

Closes #53, #54, #55, #56, #57, #58, #59

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 11:07:13 -05:00
f69d0c15a7 Merge branch 'feature/9-cleanup' into development
Sprint 9 Phase 1: TRREB Legacy Cleanup

Completed issues:
- #47: Delete legacy TRREB Python modules
- #48: Delete legacy TRREB dbt models
- #49: Remove TRREB references from Python modules
- #50: Audit and remove TRREB-related tests (none existed)
- #51: Delete legacy specification documents
- #52: Update CLAUDE.md and PROJECT_REFERENCE.md

This completes the cleanup phase of the neighbourhood dashboard transition.
2026-01-16 10:31:35 -05:00
81993b23a7 docs: Update CLAUDE.md and PROJECT_REFERENCE.md for neighbourhood transition
CLAUDE.md:
- Update project status to Sprint 9
- Remove TRREB references from data model section
- Update star schema to reflect current tables
- Simplify deferred features section
- Update reference documents

PROJECT_REFERENCE.md:
- Update import examples to neighbourhood-based
- Update data sources for neighbourhood dashboard
- Update geographic reality diagram
- Update star schema
- Modernize sprint overview
- Update scope boundaries
- Update success criteria with completed milestones
- Update reference documents

Closes #52

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 10:17:22 -05:00
457efec77f chore: Delete legacy specification documents
- Delete toronto_housing_dashboard_spec_v5.md (TRREB-based design)
- Delete wbs_sprint_plan_v4.md (outdated sprint plan)

These documents reference the deprecated TRREB-based approach.
New implementation will follow Change-Toronto-Analysis-Reviewed.md.

Closes #51

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 10:14:26 -05:00
f5f2bf3706 chore: Delete legacy TRREB dbt models
- Delete stg_trreb__purchases.sql and stg_dimensions__trreb_districts.sql
- Delete int_purchases__monthly.sql
- Delete mart_toronto_purchases.sql and mart_toronto_market_summary.sql
- Update _sources.yml to remove fact_purchases and dim_trreb_district
- Update _staging.yml to remove TRREB staging models
- Update _intermediate.yml to remove int_purchases__monthly
- Update _marts.yml to remove purchase-related marts

Closes #48

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 10:12:58 -05:00
fcaefabce8 chore: Remove TRREB references from Python modules
- Remove DimTRREBDistrict model and FactPurchases model
- Remove TRREBDistrict schema and AreaType enum
- Remove TRREBDistrictParser from geo parsers
- Remove load_trreb_districts from dimension loaders
- Remove create_district_map from choropleth figures
- Remove get_demo_districts and get_demo_purchase_data from demo_data
- Update summary metrics to remove purchase-related metrics
- Update callbacks to remove TRREB-related comments
- Update methodology page to remove TRREB data source section
- Update dashboard data notice to remove TRREB mention

Closes #49

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 10:11:11 -05:00
cb877df9e1 refactor: Delete legacy TRREB Python modules (#47)
- Delete portfolio_app/toronto/schemas/trreb.py
- Delete portfolio_app/toronto/parsers/trreb.py
- Delete portfolio_app/toronto/loaders/trreb.py
- Remove TRREB imports from __init__.py files

Part of Sprint 9: Toronto Neighbourhood Dashboard transition
See docs/changes/Change-Toronto-Analysis-Reviewed.md

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 10:00:47 -05:00
48b4eeeb62 docs: Streamline implementation plan to v2.0
- Remove code snippets (will follow existing patterns during implementation)
- Remove appendices (risk, testing, file counts)
- Remove implementation timeline (handled by sprint planning)
- Keep Phase 1 cleanup detailed (actionable)
- Reduce Phases 3-5 to reference tables
- Keep notebook template, remove premature file listings
- Add CMHC zone mapping note

Trimmed from ~420 lines to ~180 lines for execution clarity.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 09:39:40 -05:00
d3ca4ad4eb docs: Add implementation plan for neighbourhood dashboard transition
- Create comprehensive transition plan (Change-Toronto-Analysis-Reviewed.md)
  covering cleanup, new data pipeline, dbt restructuring, and dashboard tabs
- Update CLAUDE.md to reflect current app structure (Sprint 8 pages)
- Add reference to new documentation in CLAUDE.md
- Update import examples from TRREB to neighbourhood-based
- Mark legacy docs as being replaced
- Add Jupyter notebook requirements (one per graph with data reference)
- Add CRITICAL rule: NEVER delete development branch

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 09:29:57 -05:00
e7bc545f25 Merge pull request 'Change-Toronto-Analysis' (#46) from lmiranda-change-proposal into development
Reviewed-on: lmiranda/personal-portfolio#46
2026-01-16 12:52:33 +00:00
c8f4cc6241 Change-Toronto-Analysis 2026-01-16 12:52:18 +00:00
3cd2eada7c Merge pull request 'feat: Implement Sprint 8 - Portfolio website expansion (MVP)' (#45) from feature/8-portfolio-expansion into development
Reviewed-on: lmiranda/personal-portfolio#45
2026-01-15 20:48:01 +00:00
138e6fe497 feat: Implement Sprint 8 - Portfolio website expansion (MVP)
New pages:
- Home: Redesigned with hero, impact stats, featured project
- About: 6-section professional narrative
- Projects: Hub with 4 project cards and status badges
- Resume: Inline display with download placeholders
- Contact: Form UI (disabled) with contact info
- Blog: Markdown-based system with frontmatter support

Infrastructure:
- Blog system with markdown loader (python-frontmatter, markdown, pygments)
- Sidebar callback for active state highlighting on navigation
- Separated navigation into main pages and projects/dashboards groups

Closes #36, #37, #38, #39, #40, #41, #42, #43

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-15 15:40:01 -05:00
cd7b5ce154 Merge branch 'development' of ssh://hotserv.tailc9b278.ts.net:2222/lmiranda/personal-portfolio into development 2026-01-15 14:22:53 -05:00
e1135a77a8 Merge pull request 'Added Change Proposal' (#35) from lmiranda-change-proposal into development
Reviewed-on: lmiranda/personal-portfolio#35
2026-01-15 19:19:47 +00:00
39656ca836 Added Change Proposal
Changing the entire page layout and disposition. It should be transformed into a proper project document.
2026-01-15 19:19:19 +00:00
d64f90b3d3 Merge branch 'feature/7-nav-theme-modernization' into development sprint-7-complete 2026-01-15 11:53:22 -05:00
b3fb94c7cb feat: Add floating sidebar navigation and dark theme support
- Add floating pill-shaped sidebar with navigation icons
- Implement dark/light theme toggle with localStorage persistence
- Update all figure factories for transparent backgrounds
- Use carto-darkmatter map style for choropleths
- Add methodology link button to Toronto dashboard header
- Add back to dashboard button on methodology page
- Remove social links from home page (now in sidebar)
- Update CLAUDE.md to Sprint 7

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-15 11:53:13 -05:00
1e0ea9cca2 Merge pull request 'feat: Add GeoJSON parsers and choropleth map visualization' (#26) from feature/geo-parsers-choropleth into development 2026-01-14 23:02:21 +00:00
9dfa24fb76 feat: add GeoJSON parsers and choropleth map visualization
- Add geo.py parser module with CMHCZoneParser, TRREBDistrictParser,
  and NeighbourhoodParser for loading geographic boundaries
- Add coordinate reprojection support (EPSG:3857 to WGS84)
- Organize geo data in data/toronto/raw/geo/ directory
- Add CMHC zones GeoJSON (31 zones) for rental market choropleth
- Add Toronto neighbourhoods GeoJSON (158) as purchase market proxy
- Update callbacks with real CMHC 2024 rental data
- Add sample purchase data for all 158 neighbourhoods
- Update pre-commit config to exclude geo data files

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-14 17:58:13 -05:00
8701a12b41 Merge pull request 'Upload files to "/"' (#24) from lmiranda-cmhc-zones into development
Reviewed-on: lmiranda/personal-portfolio#24
2026-01-14 21:04:24 +00:00
6ef5460ad0 Upload files to "/" 2026-01-14 21:04:06 +00:00
19ffc04573 Merge pull request 'fix: Toronto page registration for Dash Pages' (#23) from fix/toronto-page-registration into development 2026-01-12 03:19:49 +00:00
08aa61f85e fix: rename Toronto page __init__.py to dashboard.py for Dash Pages
Dash Pages does not auto-discover __init__.py files as page modules.
Renamed to dashboard.py so the page registers correctly at /toronto.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 22:08:49 -05:00
2a6db2a252 Merge pull request 'feat: Sprint 6 - Polish and deployment preparation' (#22) from feature/sprint6-polish-deploy into development 2026-01-12 02:51:14 +00:00
140d3085bf feat: Sprint 6 polish - methodology, demo data, deployment prep
- Add policy event markers to time series charts
- Create methodology page (/toronto/methodology) with data sources
- Add demo data module for testing without full pipeline
- Update README with project documentation
- Add health check endpoint (/health)
- Add database initialization script
- Export new figure factory functions

Closes #21

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 21:50:45 -05:00
ad6ee3d37f Merge pull request 'feat: Sprint 5 - Visualization' (#19) from feature/sprint5-visualization into development 2026-01-11 21:22:59 +00:00
077e426d34 feat: add Sprint 5 visualization components and Toronto dashboard
- Add figure factories: choropleth, time_series, summary_cards
- Add shared components: map_controls, time_slider, metric_card
- Create Toronto dashboard page with KPI cards, choropleth maps, and time series
- Add dashboard callbacks for interactivity
- Placeholder data for demonstration until QGIS boundaries are complete

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 16:20:01 -05:00
b7907e68e4 Merge pull request 'feat: Sprint 4 - Loaders and dbt models' (#17) from feature/sprint4-loaders-dbt into development 2026-01-11 21:08:01 +00:00
457bb49395 feat: add loaders and dbt models for Toronto housing data
Sprint 4 implementation:

Loaders:
- base.py: Session management, bulk insert, upsert utilities
- dimensions.py: Load time, district, zone, neighbourhood, policy dimensions
- trreb.py: Load TRREB purchase data to fact_purchases
- cmhc.py: Load CMHC rental data to fact_rentals

dbt Project:
- Project configuration (dbt_project.yml, packages.yml)
- Staging models for all fact and dimension tables
- Intermediate models with dimension enrichment
- Marts: purchase analysis, rental analysis, market summary

Closes #16

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 16:07:30 -05:00
88e23674a8 Merge pull request 'data: add TRREB and CMHC raw data files' (#15) from data/raw-data-files into development 2026-01-11 21:01:15 +00:00
1c42533834 data: add TRREB and CMHC raw data files
- TRREB Market Watch PDFs (2024-2025, 24 files)
- CMHC Rental Market Survey Excel files (2021-2025, 5 files)
- Update pre-commit to exclude data/raw/ from large file check

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 15:58:32 -05:00
802efab8b8 Merge pull request 'feat: Sprint 3 - Pydantic schemas, SQLAlchemy models, and parser structure' (#14) from feature/sprint3-schemas-models into development 2026-01-11 20:00:20 +00:00
ead6d91a28 feat: add Pydantic schemas, SQLAlchemy models, and parser structure
Sprint 3 implementation:
- Pydantic schemas for TRREB, CMHC, and dimension data validation
- SQLAlchemy models with PostGIS geometry for fact and dimension tables
- Parser structure (stubs) for TRREB PDF and CMHC CSV processing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 14:58:31 -05:00
549e1fcbaf Merge pull request 'feat: implement bio landing page with dash-mantine-components' (#12) from feature/sprint2-bio-page into development 2026-01-11 19:44:23 +00:00
3ee4c20f5e feat: implement bio landing page with dash-mantine-components
- Full bio page with hero, summary, tech stack, projects, social links
- MantineProvider theme integration in app.py
- Responsive layout using DMC SimpleGrid
- Added dash-iconify for social link icons
- Updated mypy overrides for DMC/iconify modules

Closes #11

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 14:43:50 -05:00
68cc5bbe66 Merge pull request 'Upload files to "docs"' (#10) from lmiranda-patch-bio-doc-added into development
Reviewed-on: lmiranda/personal-portfolio#10
2026-01-11 19:38:17 +00:00
58f2c692e3 Upload files to "docs" 2026-01-11 19:38:03 +00:00
8200bbaa99 Merge pull request 'fix: update all dependencies to current versions' (#9) from fix/update-dependencies into development 2026-01-11 19:31:20 +00:00
15da8a97ce fix: update all dependencies to current versions
Updated to January 2026 versions:
- dash: 3.3+
- plotly: 6.5+
- dash-mantine-components: 2.4+
- pandas: 2.3+
- geopandas: 1.1+
- sqlalchemy: 2.0.45+
- pydantic: 2.10+
- pytest: 8.3+
- ruff: 0.8+
- mypy: 1.14+
- dbt-postgres: 1.9+

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 14:29:15 -05:00
eb01ad1101 Merge pull request 'feat: add app foundation (config.py, app.py, home page)' (#8) from feature/sprint1-app-foundation into development 2026-01-11 19:09:43 +00:00
8453f78e31 feat: add app foundation (config.py, app.py, home page)
- config.py: Pydantic BaseSettings for env loading
- app.py: Dash factory with Pages routing
- pages/home.py: Placeholder landing page

Closes #7

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 14:09:28 -05:00
ff0f5a9b51 Merge pull request 'docs: update CLAUDE.md reference doc paths' (#6) from fix/claude-md-refs into development 2026-01-11 19:03:03 +00:00
10f46f7cf1 docs: update CLAUDE.md reference doc paths
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 14:02:39 -05:00
160dc90308 Merge pull request 'Upload files to "docs"' (#5) from lmiranda-patch-project-reference-docs into development
Reviewed-on: lmiranda/personal-portfolio#5
2026-01-11 19:02:00 +00:00
ff58e0a3ea Upload files to "docs" 2026-01-11 19:01:43 +00:00
38e4a0354b Merge pull request 'feat: project bootstrap and structure' (#4) from feature/sprint1-bootstrap into development 2026-01-11 18:58:42 +00:00
c7e9b88adb feat: project bootstrap and structure
Sprint 1 initialization:
- Project directory structure (portfolio_app/, tests/, dbt/, data/, scripts/)
- CLAUDE.md with AI assistant context
- pyproject.toml with all dependencies
- docker-compose.yml for PostgreSQL 16 + PostGIS
- Makefile with standard targets
- Pre-commit configuration (ruff, mypy)
- Environment template (.env.example)
- Error handling foundation (PortfolioError hierarchy)
- Test configuration (conftest.py, pytest config)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 13:49:28 -05:00
01a0984333 Merge pull request 'uploaded initial documentation.' (#2) from init-setup into development
Reviewed-on: lmiranda/personal-portfolio#2
2026-01-11 18:41:21 +00:00