refactor: multi-dashboard structural migration
Some checks failed
CI / lint-and-test (pull_request) Has been cancelled
Some checks failed
CI / lint-and-test (pull_request) Has been cancelled
- Rename dbt project from toronto_housing to portfolio - Restructure dbt models into domain subdirectories: - shared/ for cross-domain dimensions (dim_time) - staging/toronto/, intermediate/toronto/, marts/toronto/ - Update SQLAlchemy models for raw_toronto schema - Add explicit cross-schema FK relationships for FactRentals - Namespace figure factories under figures/toronto/ - Namespace notebooks under notebooks/toronto/ - Update Makefile with domain-specific targets and env loading - Update all documentation for multi-dashboard structure This enables adding new dashboard projects (e.g., /football, /energy) without structural conflicts or naming collisions. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
79
CLAUDE.md
79
CLAUDE.md
@@ -1,5 +1,37 @@
|
||||
# CLAUDE.md
|
||||
|
||||
## ⛔ MANDATORY BEHAVIOR RULES - READ FIRST
|
||||
|
||||
**These rules are NON-NEGOTIABLE. Violating them wastes the user's time and money.**
|
||||
|
||||
### 1. WHEN USER ASKS YOU TO CHECK SOMETHING - CHECK EVERYTHING
|
||||
- Search ALL locations, not just where you think it is
|
||||
- Check cache directories: `~/.claude/plugins/cache/`
|
||||
- Check installed: `~/.claude/plugins/marketplaces/`
|
||||
- Check source directories
|
||||
- **NEVER say "no" or "that's not the issue" without exhaustive verification**
|
||||
|
||||
### 2. WHEN USER SAYS SOMETHING IS WRONG - BELIEVE THEM
|
||||
- The user knows their system better than you
|
||||
- Investigate thoroughly before disagreeing
|
||||
- **Your confidence is often wrong. User's instincts are often right.**
|
||||
|
||||
### 3. NEVER SAY "DONE" WITHOUT VERIFICATION
|
||||
- Run the actual command/script to verify
|
||||
- Show the output to the user
|
||||
- **"Done" means VERIFIED WORKING, not "I made changes"**
|
||||
|
||||
### 4. SHOW EXACTLY WHAT USER ASKS FOR
|
||||
- If user asks for messages, show the MESSAGES
|
||||
- If user asks for code, show the CODE
|
||||
- **Do not interpret or summarize unless asked**
|
||||
|
||||
**FAILURE TO FOLLOW THESE RULES = WASTED USER TIME = UNACCEPTABLE**
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
Working context for Claude Code on the Analytics Portfolio project.
|
||||
|
||||
---
|
||||
@@ -26,8 +58,9 @@ make db-init # Initialize database schema
|
||||
make db-reset # Drop and recreate database (DESTRUCTIVE)
|
||||
|
||||
# Data Loading
|
||||
make load-data # Load Toronto data from APIs, seed dev data
|
||||
make load-data-only # Load Toronto data without dbt or seeding
|
||||
make load-data # Load all project data (currently: Toronto)
|
||||
make load-toronto # Load Toronto data from APIs
|
||||
make load-toronto-only # Load Toronto data without dbt or seeding
|
||||
make seed-data # Seed sample development data
|
||||
|
||||
# Application
|
||||
@@ -127,13 +160,21 @@ class LoadError(PortfolioError):
|
||||
| `pages/` | Dash Pages (file-based routing) | URLs match file paths |
|
||||
| `pages/toronto/` | Toronto Dashboard | `tabs/` for layouts, `callbacks/` for interactions |
|
||||
| `components/` | Shared UI components | metric_card, sidebar, map_controls, time_slider |
|
||||
| `figures/` | Plotly chart factories | choropleth, bar_charts, scatter, radar, time_series |
|
||||
| `figures/toronto/` | Toronto chart factories | choropleth, bar_charts, scatter, radar, time_series |
|
||||
| `toronto/` | Toronto data logic | parsers/, loaders/, schemas/, models/ |
|
||||
| `content/blog/` | Markdown blog articles | Processed by `utils/markdown_loader.py` |
|
||||
| `notebooks/` | Data documentation | 5 domains: overview, housing, safety, demographics, amenities |
|
||||
| `notebooks/toronto/` | Toronto documentation | 5 domains: overview, housing, safety, demographics, amenities |
|
||||
|
||||
**Key URLs:** `/` (home), `/toronto` (dashboard), `/blog` (listing), `/blog/{slug}` (articles)
|
||||
|
||||
### Multi-Dashboard Architecture
|
||||
|
||||
The codebase is structured to support multiple dashboard projects:
|
||||
- **figures/**: Domain-namespaced figure factories (`figures/toronto/`, future: `figures/football/`)
|
||||
- **notebooks/**: Domain-namespaced documentation (`notebooks/toronto/`, future: `notebooks/football/`)
|
||||
- **dbt models**: Domain subdirectories (`staging/toronto/`, `marts/toronto/`)
|
||||
- **Database schemas**: Domain-specific raw data (`raw_toronto`, future: `raw_football`)
|
||||
|
||||
---
|
||||
|
||||
## Tech Stack (Locked)
|
||||
@@ -161,6 +202,16 @@ class LoadError(PortfolioError):
|
||||
|
||||
## Data Model Overview
|
||||
|
||||
### Database Schemas
|
||||
|
||||
| Schema | Purpose |
|
||||
|--------|---------|
|
||||
| `public` | Shared dimensions (dim_time) |
|
||||
| `raw_toronto` | Toronto-specific raw/dimension tables |
|
||||
| `staging` | dbt staging views |
|
||||
| `intermediate` | dbt intermediate views |
|
||||
| `marts` | dbt mart tables |
|
||||
|
||||
### Geographic Reality (Toronto Housing)
|
||||
|
||||
```
|
||||
@@ -168,20 +219,31 @@ City Neighbourhoods (158) - Primary geographic unit for analysis
|
||||
CMHC Zones (~20) - Rental data (Census Tract aligned)
|
||||
```
|
||||
|
||||
### Star Schema
|
||||
### Star Schema (raw_toronto)
|
||||
|
||||
| Table | Type | Keys |
|
||||
|-------|------|------|
|
||||
| `fact_rentals` | Fact | -> dim_time, dim_cmhc_zone |
|
||||
| `dim_time` | Dimension | date_key (PK) |
|
||||
| `dim_time` | Dimension (public) | date_key (PK) - shared |
|
||||
| `dim_cmhc_zone` | Dimension | zone_key (PK), geometry |
|
||||
| `dim_neighbourhood` | Dimension | neighbourhood_id (PK), geometry |
|
||||
| `dim_policy_event` | Dimension | event_id (PK) |
|
||||
|
||||
### dbt Layers
|
||||
### dbt Project: `portfolio`
|
||||
|
||||
**Model Structure:**
|
||||
```
|
||||
dbt/models/
|
||||
├── shared/ # Cross-domain dimensions
|
||||
│ └── stg_dimensions__time.sql
|
||||
├── staging/toronto/ # Toronto staging models
|
||||
├── intermediate/toronto/ # Toronto intermediate models
|
||||
└── marts/toronto/ # Toronto mart tables
|
||||
```
|
||||
|
||||
| Layer | Naming | Purpose |
|
||||
|-------|--------|---------|
|
||||
| Shared | `stg_dimensions__*` | Cross-domain dimensions |
|
||||
| Staging | `stg_{source}__{entity}` | 1:1 source, cleaned, typed |
|
||||
| Intermediate | `int_{domain}__{transform}` | Business logic |
|
||||
| Marts | `mart_{domain}` | Final analytical tables |
|
||||
@@ -196,7 +258,6 @@ CMHC Zones (~20) - Rental data (Census Tract aligned)
|
||||
|---------|--------|
|
||||
| Historical boundary reconciliation (140->158) | 2021+ data only for V1 |
|
||||
| ML prediction models | Energy project scope (future phase) |
|
||||
| Multi-project shared infrastructure | Build first, abstract second |
|
||||
|
||||
---
|
||||
|
||||
@@ -351,4 +412,4 @@ Use for git operations assistance.
|
||||
|
||||
---
|
||||
|
||||
*Last Updated: January 2026 (Post-Sprint 9)*
|
||||
*Last Updated: February 2026 (Multi-Dashboard Architecture)*
|
||||
|
||||
Reference in New Issue
Block a user