refactor: multi-dashboard structural migration
Some checks failed
CI / lint-and-test (pull_request) Has been cancelled

- Rename dbt project from toronto_housing to portfolio
- Restructure dbt models into domain subdirectories:
  - shared/ for cross-domain dimensions (dim_time)
  - staging/toronto/, intermediate/toronto/, marts/toronto/
- Update SQLAlchemy models for raw_toronto schema
- Add explicit cross-schema FK relationships for FactRentals
- Namespace figure factories under figures/toronto/
- Namespace notebooks under notebooks/toronto/
- Update Makefile with domain-specific targets and env loading
- Update all documentation for multi-dashboard structure

This enables adding new dashboard projects (e.g., /football, /energy)
without structural conflicts or naming collisions.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
2026-02-01 19:08:20 -05:00
parent a5d6866d63
commit 62d1a52eed
73 changed files with 1114 additions and 623 deletions

View File

@@ -1,5 +1,37 @@
# CLAUDE.md
## ⛔ MANDATORY BEHAVIOR RULES - READ FIRST
**These rules are NON-NEGOTIABLE. Violating them wastes the user's time and money.**
### 1. WHEN USER ASKS YOU TO CHECK SOMETHING - CHECK EVERYTHING
- Search ALL locations, not just where you think it is
- Check cache directories: `~/.claude/plugins/cache/`
- Check installed: `~/.claude/plugins/marketplaces/`
- Check source directories
- **NEVER say "no" or "that's not the issue" without exhaustive verification**
### 2. WHEN USER SAYS SOMETHING IS WRONG - BELIEVE THEM
- The user knows their system better than you
- Investigate thoroughly before disagreeing
- **Your confidence is often wrong. User's instincts are often right.**
### 3. NEVER SAY "DONE" WITHOUT VERIFICATION
- Run the actual command/script to verify
- Show the output to the user
- **"Done" means VERIFIED WORKING, not "I made changes"**
### 4. SHOW EXACTLY WHAT USER ASKS FOR
- If user asks for messages, show the MESSAGES
- If user asks for code, show the CODE
- **Do not interpret or summarize unless asked**
**FAILURE TO FOLLOW THESE RULES = WASTED USER TIME = UNACCEPTABLE**
---
Working context for Claude Code on the Analytics Portfolio project.
---
@@ -26,8 +58,9 @@ make db-init # Initialize database schema
make db-reset # Drop and recreate database (DESTRUCTIVE)
# Data Loading
make load-data # Load Toronto data from APIs, seed dev data
make load-data-only # Load Toronto data without dbt or seeding
make load-data # Load all project data (currently: Toronto)
make load-toronto # Load Toronto data from APIs
make load-toronto-only # Load Toronto data without dbt or seeding
make seed-data # Seed sample development data
# Application
@@ -127,13 +160,21 @@ class LoadError(PortfolioError):
| `pages/` | Dash Pages (file-based routing) | URLs match file paths |
| `pages/toronto/` | Toronto Dashboard | `tabs/` for layouts, `callbacks/` for interactions |
| `components/` | Shared UI components | metric_card, sidebar, map_controls, time_slider |
| `figures/` | Plotly chart factories | choropleth, bar_charts, scatter, radar, time_series |
| `figures/toronto/` | Toronto chart factories | choropleth, bar_charts, scatter, radar, time_series |
| `toronto/` | Toronto data logic | parsers/, loaders/, schemas/, models/ |
| `content/blog/` | Markdown blog articles | Processed by `utils/markdown_loader.py` |
| `notebooks/` | Data documentation | 5 domains: overview, housing, safety, demographics, amenities |
| `notebooks/toronto/` | Toronto documentation | 5 domains: overview, housing, safety, demographics, amenities |
**Key URLs:** `/` (home), `/toronto` (dashboard), `/blog` (listing), `/blog/{slug}` (articles)
### Multi-Dashboard Architecture
The codebase is structured to support multiple dashboard projects:
- **figures/**: Domain-namespaced figure factories (`figures/toronto/`, future: `figures/football/`)
- **notebooks/**: Domain-namespaced documentation (`notebooks/toronto/`, future: `notebooks/football/`)
- **dbt models**: Domain subdirectories (`staging/toronto/`, `marts/toronto/`)
- **Database schemas**: Domain-specific raw data (`raw_toronto`, future: `raw_football`)
---
## Tech Stack (Locked)
@@ -161,6 +202,16 @@ class LoadError(PortfolioError):
## Data Model Overview
### Database Schemas
| Schema | Purpose |
|--------|---------|
| `public` | Shared dimensions (dim_time) |
| `raw_toronto` | Toronto-specific raw/dimension tables |
| `staging` | dbt staging views |
| `intermediate` | dbt intermediate views |
| `marts` | dbt mart tables |
### Geographic Reality (Toronto Housing)
```
@@ -168,20 +219,31 @@ City Neighbourhoods (158) - Primary geographic unit for analysis
CMHC Zones (~20) - Rental data (Census Tract aligned)
```
### Star Schema
### Star Schema (raw_toronto)
| Table | Type | Keys |
|-------|------|------|
| `fact_rentals` | Fact | -> dim_time, dim_cmhc_zone |
| `dim_time` | Dimension | date_key (PK) |
| `dim_time` | Dimension (public) | date_key (PK) - shared |
| `dim_cmhc_zone` | Dimension | zone_key (PK), geometry |
| `dim_neighbourhood` | Dimension | neighbourhood_id (PK), geometry |
| `dim_policy_event` | Dimension | event_id (PK) |
### dbt Layers
### dbt Project: `portfolio`
**Model Structure:**
```
dbt/models/
├── shared/ # Cross-domain dimensions
│ └── stg_dimensions__time.sql
├── staging/toronto/ # Toronto staging models
├── intermediate/toronto/ # Toronto intermediate models
└── marts/toronto/ # Toronto mart tables
```
| Layer | Naming | Purpose |
|-------|--------|---------|
| Shared | `stg_dimensions__*` | Cross-domain dimensions |
| Staging | `stg_{source}__{entity}` | 1:1 source, cleaned, typed |
| Intermediate | `int_{domain}__{transform}` | Business logic |
| Marts | `mart_{domain}` | Final analytical tables |
@@ -196,7 +258,6 @@ CMHC Zones (~20) - Rental data (Census Tract aligned)
|---------|--------|
| Historical boundary reconciliation (140->158) | 2021+ data only for V1 |
| ML prediction models | Energy project scope (future phase) |
| Multi-project shared infrastructure | Build first, abstract second |
---
@@ -351,4 +412,4 @@ Use for git operations assistance.
---
*Last Updated: January 2026 (Post-Sprint 9)*
*Last Updated: February 2026 (Multi-Dashboard Architecture)*