feat: Complete Phase 5 dashboard implementation

Implement full 5-tab Toronto Neighbourhood Dashboard with real data
connectivity:

Dashboard Structure:
- Overview tab with livability scores and rankings
- Housing tab with affordability metrics
- Safety tab with crime statistics
- Demographics tab with population/income data
- Amenities tab with parks, schools, transit

Figure Factories (portfolio_app/figures/):
- bar_charts.py: ranking, stacked, horizontal bars
- scatter.py: scatter plots, bubble charts
- radar.py: spider/radar charts
- demographics.py: donut, age pyramid, income distribution

Service Layer (portfolio_app/toronto/services/):
- neighbourhood_service.py: queries dbt marts for all tab data
- geometry_service.py: generates GeoJSON from PostGIS
- Graceful error handling when database unavailable

Callbacks (portfolio_app/pages/toronto/callbacks/):
- map_callbacks.py: choropleth updates, map click handling
- chart_callbacks.py: supporting chart updates
- selection_callbacks.py: dropdown handlers, KPI updates

Data Pipeline (scripts/data/):
- load_toronto_data.py: orchestration script with CLI flags

Lessons Learned:
- Graceful error handling in service layers
- Modular callback structure for multi-tab dashboards
- Figure factory pattern for reusable charts

Closes: #64, #65, #66, #67, #68, #69, #70

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
2026-01-17 11:46:18 -05:00
parent 3054441630
commit c9cf744d84
27 changed files with 4377 additions and 1770 deletions

View File

@@ -10,6 +10,9 @@ This folder contains lessons learned from sprints and development work. These le
| Date | Sprint/Phase | Title | Tags |
|------|--------------|-------|------|
| 2026-01-17 | Sprint 9-10 | [Graceful Error Handling in Service Layers](./sprint-9-10-graceful-error-handling.md) | python, postgresql, error-handling, dash, graceful-degradation, arm64 |
| 2026-01-17 | Sprint 9-10 | [Modular Callback Structure](./sprint-9-10-modular-callback-structure.md) | dash, callbacks, architecture, python, code-organization |
| 2026-01-17 | Sprint 9-10 | [Figure Factory Pattern](./sprint-9-10-figure-factory-pattern.md) | plotly, dash, design-patterns, python, visualization |
| 2026-01-16 | Phase 4 | [dbt Test Syntax Deprecation](./phase-4-dbt-test-syntax.md) | dbt, testing, yaml, deprecation |
---

View File

@@ -0,0 +1,53 @@
# Sprint 9-10 - Figure Factory Pattern for Reusable Charts
## Context
Creating multiple chart types across 5 dashboard tabs, with consistent styling and behavior needed across all visualizations.
## Problem
Without a standardized approach, each callback would create figures inline with:
- Duplicated styling code (colors, fonts, backgrounds)
- Inconsistent hover templates
- Hard-to-maintain figure creation logic
- No reuse between tabs
## Solution
Created a `figures/` module with factory functions:
```
figures/
├── __init__.py # Exports all factories
├── choropleth.py # Map visualizations
├── bar_charts.py # ranking_bar, stacked_bar, horizontal_bar
├── scatter.py # scatter_figure, bubble_chart
├── radar.py # radar_figure, comparison_radar
└── demographics.py # age_pyramid, donut_chart
```
Factory pattern benefits:
1. **Consistent styling** - dark theme applied once
2. **Type-safe interfaces** - clear parameters for each chart type
3. **Easy testing** - factories can be unit tested with sample data
4. **Reusability** - same factory used across multiple tabs
Example factory signature:
```python
def create_ranking_bar(
data: list[dict],
name_column: str,
value_column: str,
title: str = "",
top_n: int = 5,
bottom_n: int = 5,
top_color: str = "#4CAF50",
bottom_color: str = "#F44336",
) -> go.Figure:
```
## Prevention
- **Create factories early** - before implementing callbacks
- **Design generic interfaces** - factories should work with any data matching the schema
- **Apply styling in one place** - use constants for colors, fonts
- **Test factories independently** - with synthetic data before integration
## Tags
plotly, dash, design-patterns, python, visualization, reusability, code-organization

View File

@@ -0,0 +1,34 @@
# Sprint 9-10 - Graceful Error Handling in Service Layers
## Context
Building the Toronto Neighbourhood Dashboard with a service layer that queries PostgreSQL/PostGIS dbt marts to provide data to Dash callbacks.
## Problem
Initial service layer implementation let database connection errors propagate as unhandled exceptions. When the PostGIS Docker container was unavailable (common on ARM64 systems where the x86_64 image fails), the entire dashboard would crash instead of gracefully degrading.
## Solution
Wrapped database queries in try/except blocks to return empty DataFrames/lists/dicts when the database is unavailable:
```python
def _execute_query(sql: str, params: dict | None = None) -> pd.DataFrame:
try:
engine = get_engine()
with engine.connect() as conn:
return pd.read_sql(text(sql), conn, params=params)
except Exception:
return pd.DataFrame()
```
This allows:
1. Dashboard to load and display empty states
2. Development/testing without running database
3. Graceful degradation in production
## Prevention
- **Always design service layers with graceful degradation** - assume external dependencies can fail
- **Return empty collections, not exceptions** - let UI components handle empty states
- **Test without database** - verify the app doesn't crash when DB is unavailable
- **Consider ARM64 compatibility** - PostGIS images may not support all platforms
## Tags
python, postgresql, service-layer, error-handling, dash, graceful-degradation, arm64

View File

@@ -0,0 +1,45 @@
# Sprint 9-10 - Modular Callback Structure for Multi-Tab Dashboards
## Context
Implementing a 5-tab Toronto Neighbourhood Dashboard with multiple callbacks per tab (map updates, chart updates, KPI updates, selection handling).
## Problem
Initial callback implementation approach would have placed all callbacks in a single file, leading to:
- A monolithic file with 500+ lines
- Difficult-to-navigate code
- Callbacks for different tabs interleaved
- Testing difficulties
## Solution
Organized callbacks into three focused modules:
```
callbacks/
├── __init__.py # Imports all modules to register callbacks
├── map_callbacks.py # Choropleth updates, map click handling
├── chart_callbacks.py # Supporting chart updates (scatter, trend, donut)
└── selection_callbacks.py # Dropdown population, KPI updates
```
Key patterns:
1. **Group by responsibility**, not by tab - all map-related callbacks together
2. **Use noqa comments** for imports that register callbacks as side effects
3. **Share helper functions** (like `_empty_chart()`) within modules
```python
# callbacks/__init__.py
from . import (
chart_callbacks, # noqa: F401
map_callbacks, # noqa: F401
selection_callbacks, # noqa: F401
)
```
## Prevention
- **Plan callback organization before implementation** - sketch which callbacks go where
- **Group by function, not by feature** - keeps related logic together
- **Keep modules under 400 lines** - split if exceeding
- **Test imports early** - verify callbacks register correctly
## Tags
dash, callbacks, architecture, python, code-organization, maintainability