feat: Complete Phase 5 dashboard implementation
Implement full 5-tab Toronto Neighbourhood Dashboard with real data connectivity: Dashboard Structure: - Overview tab with livability scores and rankings - Housing tab with affordability metrics - Safety tab with crime statistics - Demographics tab with population/income data - Amenities tab with parks, schools, transit Figure Factories (portfolio_app/figures/): - bar_charts.py: ranking, stacked, horizontal bars - scatter.py: scatter plots, bubble charts - radar.py: spider/radar charts - demographics.py: donut, age pyramid, income distribution Service Layer (portfolio_app/toronto/services/): - neighbourhood_service.py: queries dbt marts for all tab data - geometry_service.py: generates GeoJSON from PostGIS - Graceful error handling when database unavailable Callbacks (portfolio_app/pages/toronto/callbacks/): - map_callbacks.py: choropleth updates, map click handling - chart_callbacks.py: supporting chart updates - selection_callbacks.py: dropdown handlers, KPI updates Data Pipeline (scripts/data/): - load_toronto_data.py: orchestration script with CLI flags Lessons Learned: - Graceful error handling in service layers - Modular callback structure for multi-tab dashboards - Figure factory pattern for reusable charts Closes: #64, #65, #66, #67, #68, #69, #70 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -10,6 +10,9 @@ This folder contains lessons learned from sprints and development work. These le
|
||||
|
||||
| Date | Sprint/Phase | Title | Tags |
|
||||
|------|--------------|-------|------|
|
||||
| 2026-01-17 | Sprint 9-10 | [Graceful Error Handling in Service Layers](./sprint-9-10-graceful-error-handling.md) | python, postgresql, error-handling, dash, graceful-degradation, arm64 |
|
||||
| 2026-01-17 | Sprint 9-10 | [Modular Callback Structure](./sprint-9-10-modular-callback-structure.md) | dash, callbacks, architecture, python, code-organization |
|
||||
| 2026-01-17 | Sprint 9-10 | [Figure Factory Pattern](./sprint-9-10-figure-factory-pattern.md) | plotly, dash, design-patterns, python, visualization |
|
||||
| 2026-01-16 | Phase 4 | [dbt Test Syntax Deprecation](./phase-4-dbt-test-syntax.md) | dbt, testing, yaml, deprecation |
|
||||
|
||||
---
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
# Sprint 9-10 - Figure Factory Pattern for Reusable Charts
|
||||
|
||||
## Context
|
||||
Creating multiple chart types across 5 dashboard tabs, with consistent styling and behavior needed across all visualizations.
|
||||
|
||||
## Problem
|
||||
Without a standardized approach, each callback would create figures inline with:
|
||||
- Duplicated styling code (colors, fonts, backgrounds)
|
||||
- Inconsistent hover templates
|
||||
- Hard-to-maintain figure creation logic
|
||||
- No reuse between tabs
|
||||
|
||||
## Solution
|
||||
Created a `figures/` module with factory functions:
|
||||
|
||||
```
|
||||
figures/
|
||||
├── __init__.py # Exports all factories
|
||||
├── choropleth.py # Map visualizations
|
||||
├── bar_charts.py # ranking_bar, stacked_bar, horizontal_bar
|
||||
├── scatter.py # scatter_figure, bubble_chart
|
||||
├── radar.py # radar_figure, comparison_radar
|
||||
└── demographics.py # age_pyramid, donut_chart
|
||||
```
|
||||
|
||||
Factory pattern benefits:
|
||||
1. **Consistent styling** - dark theme applied once
|
||||
2. **Type-safe interfaces** - clear parameters for each chart type
|
||||
3. **Easy testing** - factories can be unit tested with sample data
|
||||
4. **Reusability** - same factory used across multiple tabs
|
||||
|
||||
Example factory signature:
|
||||
```python
|
||||
def create_ranking_bar(
|
||||
data: list[dict],
|
||||
name_column: str,
|
||||
value_column: str,
|
||||
title: str = "",
|
||||
top_n: int = 5,
|
||||
bottom_n: int = 5,
|
||||
top_color: str = "#4CAF50",
|
||||
bottom_color: str = "#F44336",
|
||||
) -> go.Figure:
|
||||
```
|
||||
|
||||
## Prevention
|
||||
- **Create factories early** - before implementing callbacks
|
||||
- **Design generic interfaces** - factories should work with any data matching the schema
|
||||
- **Apply styling in one place** - use constants for colors, fonts
|
||||
- **Test factories independently** - with synthetic data before integration
|
||||
|
||||
## Tags
|
||||
plotly, dash, design-patterns, python, visualization, reusability, code-organization
|
||||
@@ -0,0 +1,34 @@
|
||||
# Sprint 9-10 - Graceful Error Handling in Service Layers
|
||||
|
||||
## Context
|
||||
Building the Toronto Neighbourhood Dashboard with a service layer that queries PostgreSQL/PostGIS dbt marts to provide data to Dash callbacks.
|
||||
|
||||
## Problem
|
||||
Initial service layer implementation let database connection errors propagate as unhandled exceptions. When the PostGIS Docker container was unavailable (common on ARM64 systems where the x86_64 image fails), the entire dashboard would crash instead of gracefully degrading.
|
||||
|
||||
## Solution
|
||||
Wrapped database queries in try/except blocks to return empty DataFrames/lists/dicts when the database is unavailable:
|
||||
|
||||
```python
|
||||
def _execute_query(sql: str, params: dict | None = None) -> pd.DataFrame:
|
||||
try:
|
||||
engine = get_engine()
|
||||
with engine.connect() as conn:
|
||||
return pd.read_sql(text(sql), conn, params=params)
|
||||
except Exception:
|
||||
return pd.DataFrame()
|
||||
```
|
||||
|
||||
This allows:
|
||||
1. Dashboard to load and display empty states
|
||||
2. Development/testing without running database
|
||||
3. Graceful degradation in production
|
||||
|
||||
## Prevention
|
||||
- **Always design service layers with graceful degradation** - assume external dependencies can fail
|
||||
- **Return empty collections, not exceptions** - let UI components handle empty states
|
||||
- **Test without database** - verify the app doesn't crash when DB is unavailable
|
||||
- **Consider ARM64 compatibility** - PostGIS images may not support all platforms
|
||||
|
||||
## Tags
|
||||
python, postgresql, service-layer, error-handling, dash, graceful-degradation, arm64
|
||||
@@ -0,0 +1,45 @@
|
||||
# Sprint 9-10 - Modular Callback Structure for Multi-Tab Dashboards
|
||||
|
||||
## Context
|
||||
Implementing a 5-tab Toronto Neighbourhood Dashboard with multiple callbacks per tab (map updates, chart updates, KPI updates, selection handling).
|
||||
|
||||
## Problem
|
||||
Initial callback implementation approach would have placed all callbacks in a single file, leading to:
|
||||
- A monolithic file with 500+ lines
|
||||
- Difficult-to-navigate code
|
||||
- Callbacks for different tabs interleaved
|
||||
- Testing difficulties
|
||||
|
||||
## Solution
|
||||
Organized callbacks into three focused modules:
|
||||
|
||||
```
|
||||
callbacks/
|
||||
├── __init__.py # Imports all modules to register callbacks
|
||||
├── map_callbacks.py # Choropleth updates, map click handling
|
||||
├── chart_callbacks.py # Supporting chart updates (scatter, trend, donut)
|
||||
└── selection_callbacks.py # Dropdown population, KPI updates
|
||||
```
|
||||
|
||||
Key patterns:
|
||||
1. **Group by responsibility**, not by tab - all map-related callbacks together
|
||||
2. **Use noqa comments** for imports that register callbacks as side effects
|
||||
3. **Share helper functions** (like `_empty_chart()`) within modules
|
||||
|
||||
```python
|
||||
# callbacks/__init__.py
|
||||
from . import (
|
||||
chart_callbacks, # noqa: F401
|
||||
map_callbacks, # noqa: F401
|
||||
selection_callbacks, # noqa: F401
|
||||
)
|
||||
```
|
||||
|
||||
## Prevention
|
||||
- **Plan callback organization before implementation** - sketch which callbacks go where
|
||||
- **Group by function, not by feature** - keeps related logic together
|
||||
- **Keep modules under 400 lines** - split if exceeding
|
||||
- **Test imports early** - verify callbacks register correctly
|
||||
|
||||
## Tags
|
||||
dash, callbacks, architecture, python, code-organization, maintainability
|
||||
Reference in New Issue
Block a user