personal-portfolio/CLAUDE.md

CLAUDE.md

Working context for Claude Code on the Analytics Portfolio project.

---

Project Status

Current Sprint: 8 (Portfolio Website Expansion - Complete) Next Sprint: 9 (Neighbourhood Dashboard Transition) Phase: Transitioning to Toronto Neighbourhood Dashboard Branch: development (feature branches merge here)

---

Quick Reference

Run Commands

make setup          # Install deps, create .env, init pre-commit
make docker-up      # Start PostgreSQL + PostGIS
make docker-down    # Stop containers
make db-init        # Initialize database schema
make run            # Start Dash dev server
make test           # Run pytest
make lint           # Run ruff linter
make format         # Run ruff formatter
make ci             # Run all checks

Branch Workflow

1. Create feature branch FROM development: git checkout -b feature/{sprint}-{description}
2. Work and commit on feature branch
3. Merge INTO development when complete
4. Delete the feature branch after merge (keep branches clean)
5. development -> staging -> main for releases

CRITICAL: NEVER DELETE the development branch. It is the main integration branch.

---

Code Conventions

Import Style

Context	Style	Example
Same directory	Single dot	from .neighbourhood import NeighbourhoodRecord
Sibling directory	Double dot	from ..schemas.neighbourhood import CensusRecord
External packages	Absolute	import pandas as pd

Module Responsibilities

Directory	Contains	Purpose
schemas/	Pydantic models	Data validation
models/	SQLAlchemy ORM	Database persistence
parsers/	API/CSV extraction	Raw data ingestion
loaders/	Database operations	Data loading
figures/	Chart factories	Plotly figure generation
callbacks/	Dash callbacks	In pages/{dashboard}/callbacks/
errors/	Exceptions + handlers	Error handling

Type Hints

Use Python 3.10+ style:

def process(items: list[str], config: dict[str, int] | None = None) -> bool:
    ...

Error Handling

# errors/exceptions.py
class PortfolioError(Exception):
    """Base exception."""

class ParseError(PortfolioError):
    """PDF/CSV parsing failed."""

class ValidationError(PortfolioError):
    """Pydantic or business rule validation failed."""

class LoadError(PortfolioError):
    """Database load operation failed."""

Code Standards

• Single responsibility functions with verb naming
• Early returns over deep nesting
• Google-style docstrings only for non-obvious behavior
• Module-level constants for magic values
• Pydantic BaseSettings for runtime config

---

Application Structure

portfolio_app/
├── app.py                    # Dash app factory with Pages routing
├── config.py                 # Pydantic BaseSettings
├── assets/                   # CSS, images (auto-served)
│   └── sidebar.css          # Navigation styling
├── callbacks/               # Global callbacks
│   ├── sidebar.py           # Sidebar toggle
│   └── theme.py             # Dark/light theme
├── pages/
│   ├── home.py              # Bio landing page -> /
│   ├── about.py             # About page -> /about
│   ├── contact.py           # Contact form -> /contact
│   ├── health.py            # Health endpoint -> /health
│   ├── projects.py          # Project showcase -> /projects
│   ├── resume.py            # Resume/CV -> /resume
│   ├── blog/
│   │   ├── index.py         # Blog listing -> /blog
│   │   └── article.py       # Blog article -> /blog/{slug}
│   └── toronto/
│       ├── dashboard.py     # Dashboard -> /toronto
│       ├── methodology.py   # Methodology -> /toronto/methodology
│       └── callbacks/       # Dashboard interactions
├── components/              # Shared UI (sidebar, cards, controls)
│   ├── metric_card.py       # KPI card component
│   ├── map_controls.py      # Map control panel
│   ├── sidebar.py           # Navigation sidebar
│   └── time_slider.py       # Time range selector
├── figures/                 # Shared chart factories
│   ├── choropleth.py        # Map visualizations
│   ├── summary_cards.py     # KPI figures
│   └── time_series.py       # Trend charts
├── content/                 # Markdown content
│   └── blog/                # Blog articles
├── toronto/                 # Toronto data logic
│   ├── parsers/
│   ├── loaders/
│   ├── schemas/             # Pydantic
│   ├── models/              # SQLAlchemy
│   └── demo_data.py         # Sample data
├── utils/                   # Utilities
│   └── markdown_loader.py   # Markdown processing
└── errors/

URL Routing

URL	Page	Sprint
/	Bio landing page	2
/about	About page	8
/contact	Contact form	8
/health	Health endpoint	8
/projects	Project showcase	8
/resume	Resume/CV	8
/blog	Blog listing	8
/blog/{slug}	Blog article	8
/toronto	Toronto Dashboard	6
/toronto/methodology	Dashboard methodology	6

---

Tech Stack (Locked)

Layer	Technology	Version
Database	PostgreSQL + PostGIS	16.x
Validation	Pydantic	>=2.0
ORM	SQLAlchemy	>=2.0 (2.0-style API only)
Transformation	dbt-postgres	>=1.7
Data Processing	Pandas	>=2.1
Geospatial	GeoPandas + Shapely	>=0.14
Visualization	Dash + Plotly	>=2.14
UI Components	dash-mantine-components	Latest stable
Testing	pytest	>=7.0
Python	3.11+	Via pyenv

Notes:

• SQLAlchemy 2.0 + Pydantic 2.0 only (never mix 1.x APIs)
• PostGIS extension required in database
• Docker Compose V2 format (no version field)

---

Data Model Overview

Geographic Reality (Toronto Housing)

TRREB Districts (~35)     - Purchase data (W01, C01, E01...)
CMHC Zones (~20)          - Rental data (Census Tract aligned)
City Neighbourhoods (158) - Enrichment/overlay only

Critical: These geographies do NOT align. Display as separate layers—do not force crosswalks.

Star Schema

Table	Type	Keys
fact_purchases	Fact	-> dim_time, dim_trreb_district
fact_rentals	Fact	-> dim_time, dim_cmhc_zone
dim_time	Dimension	date_key (PK)
dim_trreb_district	Dimension	district_key (PK), geometry
dim_cmhc_zone	Dimension	zone_key (PK), geometry
dim_neighbourhood	Dimension	neighbourhood_id (PK), geometry
dim_policy_event	Dimension	event_id (PK)

V1 Rule: dim_neighbourhood has NO FK to fact tables—reference overlay only.

dbt Layers

Layer	Naming	Purpose
Staging	stg_{source}__{entity}	1:1 source, cleaned, typed
Intermediate	int_{domain}__{transform}	Business logic
Marts	mart_{domain}	Final analytical tables

---

DO NOT BUILD (Phase 1)

Stop and flag if a task seems to require these:

Feature	Reason
bridge_district_neighbourhood table	Area-weighted aggregation is Phase 4
Crime data integration	Deferred to Phase 4
Historical boundary reconciliation (140->158)	2021+ data only for V1
ML prediction models	Energy project scope (Phase 3)
Multi-project shared infrastructure	Build first, abstract second (Phase 2)

---

Sprint 1 Deliverables

Category	Tasks
Bootstrap	Git init, pyproject.toml, .env.example, Makefile, CLAUDE.md
Infrastructure	Docker Compose (PostgreSQL + PostGIS), scripts/ directory
App Foundation	portfolio_app/ structure, config.py, error handling
Tests	tests/ directory, conftest.py, pytest config
Data Acquisition	Download TRREB PDFs, START boundary digitization (HUMAN task)

Human Tasks (Cannot Automate)

Task	Tool	Effort
Digitize TRREB district boundaries	QGIS	3-4 hours
Research policy events (10-20)	Manual	2-3 hours
Replace social link placeholders	Manual	5 minutes

---

Environment Variables

Required in .env:

DATABASE_URL=postgresql://user:pass@localhost:5432/portfolio
POSTGRES_USER=portfolio
POSTGRES_PASSWORD=<secure>
POSTGRES_DB=portfolio
DASH_DEBUG=true
SECRET_KEY=<random>
LOG_LEVEL=INFO

---

Script Standards

All scripts in scripts/:

• Include usage comments at top
• Idempotent where possible
• Exit codes: 0 = success, 1 = error
• Use set -euo pipefail for bash
• Log to stdout, errors to stderr

---

Reference Documents

Document	Location	Use When
Full specification	docs/PROJECT_REFERENCE.md	Architecture decisions
Data schemas (legacy)	docs/toronto_housing_dashboard_spec_v5.md	Reference only - being replaced
WBS details (legacy)	docs/wbs_sprint_plan_v4.md	Reference only - being replaced
Neighbourhood Dashboard Vision	docs/changes/Change-Toronto-Analysis.md	New dashboard specification
Implementation Plan	docs/changes/Change-Toronto-Analysis-Reviewed.md	Sprint planning, cleanup tasks

---

Pending Transition

Note: This project is transitioning from a TRREB district-based housing dashboard to a comprehensive Toronto Neighbourhood Dashboard (158 neighbourhoods). See the Implementation Plan for details on:

• Files being deprecated (TRREB parsers, schemas, loaders)
• New data sources (Toronto Open Data, Toronto Police, CMHC APIs)
• New dashboard tabs (Overview, Housing, Safety, Demographics, Amenities)

---

Last Updated: Sprint 8