feat: Complete Phase 5 dashboard implementation
Implement full 5-tab Toronto Neighbourhood Dashboard with real data connectivity: Dashboard Structure: - Overview tab with livability scores and rankings - Housing tab with affordability metrics - Safety tab with crime statistics - Demographics tab with population/income data - Amenities tab with parks, schools, transit Figure Factories (portfolio_app/figures/): - bar_charts.py: ranking, stacked, horizontal bars - scatter.py: scatter plots, bubble charts - radar.py: spider/radar charts - demographics.py: donut, age pyramid, income distribution Service Layer (portfolio_app/toronto/services/): - neighbourhood_service.py: queries dbt marts for all tab data - geometry_service.py: generates GeoJSON from PostGIS - Graceful error handling when database unavailable Callbacks (portfolio_app/pages/toronto/callbacks/): - map_callbacks.py: choropleth updates, map click handling - chart_callbacks.py: supporting chart updates - selection_callbacks.py: dropdown handlers, KPI updates Data Pipeline (scripts/data/): - load_toronto_data.py: orchestration script with CLI flags Lessons Learned: - Graceful error handling in service layers - Modular callback structure for multi-tab dashboards - Figure factory pattern for reusable charts Closes: #64, #65, #66, #67, #68, #69, #70 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
10
Makefile
10
Makefile
@@ -1,4 +1,4 @@
|
|||||||
.PHONY: setup docker-up docker-down db-init run test dbt-run dbt-test lint format ci deploy clean help
|
.PHONY: setup docker-up docker-down db-init load-data run test dbt-run dbt-test lint format ci deploy clean help
|
||||||
|
|
||||||
# Default target
|
# Default target
|
||||||
.DEFAULT_GOAL := help
|
.DEFAULT_GOAL := help
|
||||||
@@ -71,6 +71,14 @@ db-reset: ## Drop and recreate database (DESTRUCTIVE)
|
|||||||
@sleep 3
|
@sleep 3
|
||||||
$(MAKE) db-init
|
$(MAKE) db-init
|
||||||
|
|
||||||
|
load-data: ## Load Toronto data from APIs and run dbt
|
||||||
|
@echo "$(GREEN)Loading Toronto neighbourhood data...$(NC)"
|
||||||
|
$(PYTHON) scripts/data/load_toronto_data.py
|
||||||
|
|
||||||
|
load-data-only: ## Load Toronto data without running dbt
|
||||||
|
@echo "$(GREEN)Loading Toronto data (skip dbt)...$(NC)"
|
||||||
|
$(PYTHON) scripts/data/load_toronto_data.py --skip-dbt
|
||||||
|
|
||||||
# =============================================================================
|
# =============================================================================
|
||||||
# Application
|
# Application
|
||||||
# =============================================================================
|
# =============================================================================
|
||||||
|
|||||||
@@ -10,6 +10,9 @@ This folder contains lessons learned from sprints and development work. These le
|
|||||||
|
|
||||||
| Date | Sprint/Phase | Title | Tags |
|
| Date | Sprint/Phase | Title | Tags |
|
||||||
|------|--------------|-------|------|
|
|------|--------------|-------|------|
|
||||||
|
| 2026-01-17 | Sprint 9-10 | [Graceful Error Handling in Service Layers](./sprint-9-10-graceful-error-handling.md) | python, postgresql, error-handling, dash, graceful-degradation, arm64 |
|
||||||
|
| 2026-01-17 | Sprint 9-10 | [Modular Callback Structure](./sprint-9-10-modular-callback-structure.md) | dash, callbacks, architecture, python, code-organization |
|
||||||
|
| 2026-01-17 | Sprint 9-10 | [Figure Factory Pattern](./sprint-9-10-figure-factory-pattern.md) | plotly, dash, design-patterns, python, visualization |
|
||||||
| 2026-01-16 | Phase 4 | [dbt Test Syntax Deprecation](./phase-4-dbt-test-syntax.md) | dbt, testing, yaml, deprecation |
|
| 2026-01-16 | Phase 4 | [dbt Test Syntax Deprecation](./phase-4-dbt-test-syntax.md) | dbt, testing, yaml, deprecation |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -0,0 +1,53 @@
|
|||||||
|
# Sprint 9-10 - Figure Factory Pattern for Reusable Charts
|
||||||
|
|
||||||
|
## Context
|
||||||
|
Creating multiple chart types across 5 dashboard tabs, with consistent styling and behavior needed across all visualizations.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
Without a standardized approach, each callback would create figures inline with:
|
||||||
|
- Duplicated styling code (colors, fonts, backgrounds)
|
||||||
|
- Inconsistent hover templates
|
||||||
|
- Hard-to-maintain figure creation logic
|
||||||
|
- No reuse between tabs
|
||||||
|
|
||||||
|
## Solution
|
||||||
|
Created a `figures/` module with factory functions:
|
||||||
|
|
||||||
|
```
|
||||||
|
figures/
|
||||||
|
├── __init__.py # Exports all factories
|
||||||
|
├── choropleth.py # Map visualizations
|
||||||
|
├── bar_charts.py # ranking_bar, stacked_bar, horizontal_bar
|
||||||
|
├── scatter.py # scatter_figure, bubble_chart
|
||||||
|
├── radar.py # radar_figure, comparison_radar
|
||||||
|
└── demographics.py # age_pyramid, donut_chart
|
||||||
|
```
|
||||||
|
|
||||||
|
Factory pattern benefits:
|
||||||
|
1. **Consistent styling** - dark theme applied once
|
||||||
|
2. **Type-safe interfaces** - clear parameters for each chart type
|
||||||
|
3. **Easy testing** - factories can be unit tested with sample data
|
||||||
|
4. **Reusability** - same factory used across multiple tabs
|
||||||
|
|
||||||
|
Example factory signature:
|
||||||
|
```python
|
||||||
|
def create_ranking_bar(
|
||||||
|
data: list[dict],
|
||||||
|
name_column: str,
|
||||||
|
value_column: str,
|
||||||
|
title: str = "",
|
||||||
|
top_n: int = 5,
|
||||||
|
bottom_n: int = 5,
|
||||||
|
top_color: str = "#4CAF50",
|
||||||
|
bottom_color: str = "#F44336",
|
||||||
|
) -> go.Figure:
|
||||||
|
```
|
||||||
|
|
||||||
|
## Prevention
|
||||||
|
- **Create factories early** - before implementing callbacks
|
||||||
|
- **Design generic interfaces** - factories should work with any data matching the schema
|
||||||
|
- **Apply styling in one place** - use constants for colors, fonts
|
||||||
|
- **Test factories independently** - with synthetic data before integration
|
||||||
|
|
||||||
|
## Tags
|
||||||
|
plotly, dash, design-patterns, python, visualization, reusability, code-organization
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
# Sprint 9-10 - Graceful Error Handling in Service Layers
|
||||||
|
|
||||||
|
## Context
|
||||||
|
Building the Toronto Neighbourhood Dashboard with a service layer that queries PostgreSQL/PostGIS dbt marts to provide data to Dash callbacks.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
Initial service layer implementation let database connection errors propagate as unhandled exceptions. When the PostGIS Docker container was unavailable (common on ARM64 systems where the x86_64 image fails), the entire dashboard would crash instead of gracefully degrading.
|
||||||
|
|
||||||
|
## Solution
|
||||||
|
Wrapped database queries in try/except blocks to return empty DataFrames/lists/dicts when the database is unavailable:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def _execute_query(sql: str, params: dict | None = None) -> pd.DataFrame:
|
||||||
|
try:
|
||||||
|
engine = get_engine()
|
||||||
|
with engine.connect() as conn:
|
||||||
|
return pd.read_sql(text(sql), conn, params=params)
|
||||||
|
except Exception:
|
||||||
|
return pd.DataFrame()
|
||||||
|
```
|
||||||
|
|
||||||
|
This allows:
|
||||||
|
1. Dashboard to load and display empty states
|
||||||
|
2. Development/testing without running database
|
||||||
|
3. Graceful degradation in production
|
||||||
|
|
||||||
|
## Prevention
|
||||||
|
- **Always design service layers with graceful degradation** - assume external dependencies can fail
|
||||||
|
- **Return empty collections, not exceptions** - let UI components handle empty states
|
||||||
|
- **Test without database** - verify the app doesn't crash when DB is unavailable
|
||||||
|
- **Consider ARM64 compatibility** - PostGIS images may not support all platforms
|
||||||
|
|
||||||
|
## Tags
|
||||||
|
python, postgresql, service-layer, error-handling, dash, graceful-degradation, arm64
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
# Sprint 9-10 - Modular Callback Structure for Multi-Tab Dashboards
|
||||||
|
|
||||||
|
## Context
|
||||||
|
Implementing a 5-tab Toronto Neighbourhood Dashboard with multiple callbacks per tab (map updates, chart updates, KPI updates, selection handling).
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
Initial callback implementation approach would have placed all callbacks in a single file, leading to:
|
||||||
|
- A monolithic file with 500+ lines
|
||||||
|
- Difficult-to-navigate code
|
||||||
|
- Callbacks for different tabs interleaved
|
||||||
|
- Testing difficulties
|
||||||
|
|
||||||
|
## Solution
|
||||||
|
Organized callbacks into three focused modules:
|
||||||
|
|
||||||
|
```
|
||||||
|
callbacks/
|
||||||
|
├── __init__.py # Imports all modules to register callbacks
|
||||||
|
├── map_callbacks.py # Choropleth updates, map click handling
|
||||||
|
├── chart_callbacks.py # Supporting chart updates (scatter, trend, donut)
|
||||||
|
└── selection_callbacks.py # Dropdown population, KPI updates
|
||||||
|
```
|
||||||
|
|
||||||
|
Key patterns:
|
||||||
|
1. **Group by responsibility**, not by tab - all map-related callbacks together
|
||||||
|
2. **Use noqa comments** for imports that register callbacks as side effects
|
||||||
|
3. **Share helper functions** (like `_empty_chart()`) within modules
|
||||||
|
|
||||||
|
```python
|
||||||
|
# callbacks/__init__.py
|
||||||
|
from . import (
|
||||||
|
chart_callbacks, # noqa: F401
|
||||||
|
map_callbacks, # noqa: F401
|
||||||
|
selection_callbacks, # noqa: F401
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Prevention
|
||||||
|
- **Plan callback organization before implementation** - sketch which callbacks go where
|
||||||
|
- **Group by function, not by feature** - keeps related logic together
|
||||||
|
- **Keep modules under 400 lines** - split if exceeding
|
||||||
|
- **Test imports early** - verify callbacks register correctly
|
||||||
|
|
||||||
|
## Tags
|
||||||
|
dash, callbacks, architecture, python, code-organization, maintainability
|
||||||
@@ -1,9 +1,27 @@
|
|||||||
"""Plotly figure factories for data visualization."""
|
"""Plotly figure factories for data visualization."""
|
||||||
|
|
||||||
|
from .bar_charts import (
|
||||||
|
create_horizontal_bar,
|
||||||
|
create_ranking_bar,
|
||||||
|
create_stacked_bar,
|
||||||
|
)
|
||||||
from .choropleth import (
|
from .choropleth import (
|
||||||
create_choropleth_figure,
|
create_choropleth_figure,
|
||||||
create_zone_map,
|
create_zone_map,
|
||||||
)
|
)
|
||||||
|
from .demographics import (
|
||||||
|
create_age_pyramid,
|
||||||
|
create_donut_chart,
|
||||||
|
create_income_distribution,
|
||||||
|
)
|
||||||
|
from .radar import (
|
||||||
|
create_comparison_radar,
|
||||||
|
create_radar_figure,
|
||||||
|
)
|
||||||
|
from .scatter import (
|
||||||
|
create_bubble_chart,
|
||||||
|
create_scatter_figure,
|
||||||
|
)
|
||||||
from .summary_cards import create_metric_card_figure, create_summary_metrics
|
from .summary_cards import create_metric_card_figure, create_summary_metrics
|
||||||
from .time_series import (
|
from .time_series import (
|
||||||
add_policy_markers,
|
add_policy_markers,
|
||||||
@@ -26,4 +44,18 @@ __all__ = [
|
|||||||
# Summary
|
# Summary
|
||||||
"create_metric_card_figure",
|
"create_metric_card_figure",
|
||||||
"create_summary_metrics",
|
"create_summary_metrics",
|
||||||
|
# Bar charts
|
||||||
|
"create_ranking_bar",
|
||||||
|
"create_stacked_bar",
|
||||||
|
"create_horizontal_bar",
|
||||||
|
# Scatter plots
|
||||||
|
"create_scatter_figure",
|
||||||
|
"create_bubble_chart",
|
||||||
|
# Radar charts
|
||||||
|
"create_radar_figure",
|
||||||
|
"create_comparison_radar",
|
||||||
|
# Demographics
|
||||||
|
"create_age_pyramid",
|
||||||
|
"create_donut_chart",
|
||||||
|
"create_income_distribution",
|
||||||
]
|
]
|
||||||
|
|||||||
238
portfolio_app/figures/bar_charts.py
Normal file
238
portfolio_app/figures/bar_charts.py
Normal file
@@ -0,0 +1,238 @@
|
|||||||
|
"""Bar chart figure factories for dashboard visualizations."""
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pandas as pd
|
||||||
|
import plotly.express as px
|
||||||
|
import plotly.graph_objects as go
|
||||||
|
|
||||||
|
|
||||||
|
def create_ranking_bar(
|
||||||
|
data: list[dict[str, Any]],
|
||||||
|
name_column: str,
|
||||||
|
value_column: str,
|
||||||
|
title: str | None = None,
|
||||||
|
top_n: int = 10,
|
||||||
|
bottom_n: int = 10,
|
||||||
|
color_top: str = "#4CAF50",
|
||||||
|
color_bottom: str = "#F44336",
|
||||||
|
value_format: str = ",.0f",
|
||||||
|
) -> go.Figure:
|
||||||
|
"""Create horizontal bar chart showing top and bottom rankings.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
data: List of data records.
|
||||||
|
name_column: Column name for labels.
|
||||||
|
value_column: Column name for values.
|
||||||
|
title: Optional chart title.
|
||||||
|
top_n: Number of top items to show.
|
||||||
|
bottom_n: Number of bottom items to show.
|
||||||
|
color_top: Color for top performers.
|
||||||
|
color_bottom: Color for bottom performers.
|
||||||
|
value_format: Number format string for values.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Plotly Figure object.
|
||||||
|
"""
|
||||||
|
if not data:
|
||||||
|
return _create_empty_figure(title or "Rankings")
|
||||||
|
|
||||||
|
df = pd.DataFrame(data).sort_values(value_column, ascending=False)
|
||||||
|
|
||||||
|
# Get top and bottom
|
||||||
|
top_df = df.head(top_n).copy()
|
||||||
|
bottom_df = df.tail(bottom_n).copy()
|
||||||
|
|
||||||
|
top_df["group"] = "Top"
|
||||||
|
bottom_df["group"] = "Bottom"
|
||||||
|
|
||||||
|
# Combine with gap in the middle
|
||||||
|
combined = pd.concat([top_df, bottom_df])
|
||||||
|
combined["color"] = combined["group"].map(
|
||||||
|
{"Top": color_top, "Bottom": color_bottom}
|
||||||
|
)
|
||||||
|
|
||||||
|
fig = go.Figure()
|
||||||
|
|
||||||
|
# Add top bars
|
||||||
|
fig.add_trace(
|
||||||
|
go.Bar(
|
||||||
|
y=top_df[name_column],
|
||||||
|
x=top_df[value_column],
|
||||||
|
orientation="h",
|
||||||
|
marker_color=color_top,
|
||||||
|
name="Top",
|
||||||
|
text=top_df[value_column].apply(lambda x: f"{x:{value_format}}"),
|
||||||
|
textposition="auto",
|
||||||
|
hovertemplate=f"%{{y}}<br>{value_column}: %{{x:{value_format}}}<extra></extra>",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
# Add bottom bars
|
||||||
|
fig.add_trace(
|
||||||
|
go.Bar(
|
||||||
|
y=bottom_df[name_column],
|
||||||
|
x=bottom_df[value_column],
|
||||||
|
orientation="h",
|
||||||
|
marker_color=color_bottom,
|
||||||
|
name="Bottom",
|
||||||
|
text=bottom_df[value_column].apply(lambda x: f"{x:{value_format}}"),
|
||||||
|
textposition="auto",
|
||||||
|
hovertemplate=f"%{{y}}<br>{value_column}: %{{x:{value_format}}}<extra></extra>",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
fig.update_layout(
|
||||||
|
title=title,
|
||||||
|
barmode="group",
|
||||||
|
showlegend=True,
|
||||||
|
legend={"orientation": "h", "yanchor": "bottom", "y": 1.02},
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
plot_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
xaxis={"gridcolor": "rgba(128,128,128,0.2)", "title": None},
|
||||||
|
yaxis={"autorange": "reversed", "title": None},
|
||||||
|
margin={"l": 10, "r": 10, "t": 40, "b": 10},
|
||||||
|
)
|
||||||
|
|
||||||
|
return fig
|
||||||
|
|
||||||
|
|
||||||
|
def create_stacked_bar(
|
||||||
|
data: list[dict[str, Any]],
|
||||||
|
x_column: str,
|
||||||
|
value_column: str,
|
||||||
|
category_column: str,
|
||||||
|
title: str | None = None,
|
||||||
|
color_map: dict[str, str] | None = None,
|
||||||
|
show_percentages: bool = False,
|
||||||
|
) -> go.Figure:
|
||||||
|
"""Create stacked bar chart for breakdown visualizations.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
data: List of data records.
|
||||||
|
x_column: Column name for x-axis categories.
|
||||||
|
value_column: Column name for values.
|
||||||
|
category_column: Column name for stacking categories.
|
||||||
|
title: Optional chart title.
|
||||||
|
color_map: Mapping of category to color.
|
||||||
|
show_percentages: Whether to normalize to 100%.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Plotly Figure object.
|
||||||
|
"""
|
||||||
|
if not data:
|
||||||
|
return _create_empty_figure(title or "Breakdown")
|
||||||
|
|
||||||
|
df = pd.DataFrame(data)
|
||||||
|
|
||||||
|
# Default color scheme
|
||||||
|
if color_map is None:
|
||||||
|
categories = df[category_column].unique()
|
||||||
|
colors = px.colors.qualitative.Set2[: len(categories)]
|
||||||
|
color_map = dict(zip(categories, colors, strict=False))
|
||||||
|
|
||||||
|
fig = px.bar(
|
||||||
|
df,
|
||||||
|
x=x_column,
|
||||||
|
y=value_column,
|
||||||
|
color=category_column,
|
||||||
|
color_discrete_map=color_map,
|
||||||
|
barmode="stack",
|
||||||
|
text=value_column if not show_percentages else None,
|
||||||
|
)
|
||||||
|
|
||||||
|
if show_percentages:
|
||||||
|
fig.update_traces(texttemplate="%{y:.1f}%", textposition="inside")
|
||||||
|
|
||||||
|
fig.update_layout(
|
||||||
|
title=title,
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
plot_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
xaxis={"gridcolor": "rgba(128,128,128,0.2)", "title": None},
|
||||||
|
yaxis={"gridcolor": "rgba(128,128,128,0.2)", "title": None},
|
||||||
|
legend={"orientation": "h", "yanchor": "bottom", "y": 1.02},
|
||||||
|
margin={"l": 10, "r": 10, "t": 60, "b": 10},
|
||||||
|
)
|
||||||
|
|
||||||
|
return fig
|
||||||
|
|
||||||
|
|
||||||
|
def create_horizontal_bar(
|
||||||
|
data: list[dict[str, Any]],
|
||||||
|
name_column: str,
|
||||||
|
value_column: str,
|
||||||
|
title: str | None = None,
|
||||||
|
color: str = "#2196F3",
|
||||||
|
value_format: str = ",.0f",
|
||||||
|
sort: bool = True,
|
||||||
|
) -> go.Figure:
|
||||||
|
"""Create simple horizontal bar chart.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
data: List of data records.
|
||||||
|
name_column: Column name for labels.
|
||||||
|
value_column: Column name for values.
|
||||||
|
title: Optional chart title.
|
||||||
|
color: Bar color.
|
||||||
|
value_format: Number format string.
|
||||||
|
sort: Whether to sort by value descending.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Plotly Figure object.
|
||||||
|
"""
|
||||||
|
if not data:
|
||||||
|
return _create_empty_figure(title or "Bar Chart")
|
||||||
|
|
||||||
|
df = pd.DataFrame(data)
|
||||||
|
|
||||||
|
if sort:
|
||||||
|
df = df.sort_values(value_column, ascending=True)
|
||||||
|
|
||||||
|
fig = go.Figure(
|
||||||
|
go.Bar(
|
||||||
|
y=df[name_column],
|
||||||
|
x=df[value_column],
|
||||||
|
orientation="h",
|
||||||
|
marker_color=color,
|
||||||
|
text=df[value_column].apply(lambda x: f"{x:{value_format}}"),
|
||||||
|
textposition="outside",
|
||||||
|
hovertemplate=f"%{{y}}<br>Value: %{{x:{value_format}}}<extra></extra>",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
fig.update_layout(
|
||||||
|
title=title,
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
plot_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
xaxis={"gridcolor": "rgba(128,128,128,0.2)", "title": None},
|
||||||
|
yaxis={"title": None},
|
||||||
|
margin={"l": 10, "r": 10, "t": 40, "b": 10},
|
||||||
|
)
|
||||||
|
|
||||||
|
return fig
|
||||||
|
|
||||||
|
|
||||||
|
def _create_empty_figure(title: str) -> go.Figure:
|
||||||
|
"""Create an empty figure with a message."""
|
||||||
|
fig = go.Figure()
|
||||||
|
fig.add_annotation(
|
||||||
|
text="No data available",
|
||||||
|
xref="paper",
|
||||||
|
yref="paper",
|
||||||
|
x=0.5,
|
||||||
|
y=0.5,
|
||||||
|
showarrow=False,
|
||||||
|
font={"size": 14, "color": "#888888"},
|
||||||
|
)
|
||||||
|
fig.update_layout(
|
||||||
|
title=title,
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
plot_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
xaxis={"visible": False},
|
||||||
|
yaxis={"visible": False},
|
||||||
|
)
|
||||||
|
return fig
|
||||||
240
portfolio_app/figures/demographics.py
Normal file
240
portfolio_app/figures/demographics.py
Normal file
@@ -0,0 +1,240 @@
|
|||||||
|
"""Demographics-specific chart factories."""
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pandas as pd
|
||||||
|
import plotly.graph_objects as go
|
||||||
|
|
||||||
|
|
||||||
|
def create_age_pyramid(
|
||||||
|
data: list[dict[str, Any]],
|
||||||
|
age_groups: list[str],
|
||||||
|
male_column: str = "male",
|
||||||
|
female_column: str = "female",
|
||||||
|
title: str | None = None,
|
||||||
|
) -> go.Figure:
|
||||||
|
"""Create population pyramid by age and gender.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
data: List with one record per age group containing male/female counts.
|
||||||
|
age_groups: List of age group labels in order (youngest to oldest).
|
||||||
|
male_column: Column name for male population.
|
||||||
|
female_column: Column name for female population.
|
||||||
|
title: Optional chart title.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Plotly Figure object.
|
||||||
|
"""
|
||||||
|
if not data or not age_groups:
|
||||||
|
return _create_empty_figure(title or "Age Distribution")
|
||||||
|
|
||||||
|
df = pd.DataFrame(data)
|
||||||
|
|
||||||
|
# Ensure data is ordered by age groups
|
||||||
|
if "age_group" in df.columns:
|
||||||
|
df["age_order"] = df["age_group"].apply(
|
||||||
|
lambda x: age_groups.index(x) if x in age_groups else -1
|
||||||
|
)
|
||||||
|
df = df.sort_values("age_order")
|
||||||
|
|
||||||
|
male_values = df[male_column].tolist() if male_column in df.columns else []
|
||||||
|
female_values = df[female_column].tolist() if female_column in df.columns else []
|
||||||
|
|
||||||
|
# Make male values negative for pyramid effect
|
||||||
|
male_values_neg = [-v for v in male_values]
|
||||||
|
|
||||||
|
fig = go.Figure()
|
||||||
|
|
||||||
|
# Male bars (left side, negative values)
|
||||||
|
fig.add_trace(
|
||||||
|
go.Bar(
|
||||||
|
y=age_groups,
|
||||||
|
x=male_values_neg,
|
||||||
|
orientation="h",
|
||||||
|
name="Male",
|
||||||
|
marker_color="#2196F3",
|
||||||
|
hovertemplate="%{y}<br>Male: %{customdata:,}<extra></extra>",
|
||||||
|
customdata=male_values,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
# Female bars (right side, positive values)
|
||||||
|
fig.add_trace(
|
||||||
|
go.Bar(
|
||||||
|
y=age_groups,
|
||||||
|
x=female_values,
|
||||||
|
orientation="h",
|
||||||
|
name="Female",
|
||||||
|
marker_color="#E91E63",
|
||||||
|
hovertemplate="%{y}<br>Female: %{x:,}<extra></extra>",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
# Calculate max for symmetric axis
|
||||||
|
max_val = max(max(male_values, default=0), max(female_values, default=0))
|
||||||
|
|
||||||
|
fig.update_layout(
|
||||||
|
title=title,
|
||||||
|
barmode="overlay",
|
||||||
|
bargap=0.1,
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
plot_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
xaxis={
|
||||||
|
"title": "Population",
|
||||||
|
"gridcolor": "rgba(128,128,128,0.2)",
|
||||||
|
"range": [-max_val * 1.1, max_val * 1.1],
|
||||||
|
"tickvals": [-max_val, -max_val / 2, 0, max_val / 2, max_val],
|
||||||
|
"ticktext": [
|
||||||
|
f"{max_val:,.0f}",
|
||||||
|
f"{max_val / 2:,.0f}",
|
||||||
|
"0",
|
||||||
|
f"{max_val / 2:,.0f}",
|
||||||
|
f"{max_val:,.0f}",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
yaxis={"title": None, "gridcolor": "rgba(128,128,128,0.2)"},
|
||||||
|
legend={"orientation": "h", "yanchor": "bottom", "y": 1.02},
|
||||||
|
margin={"l": 10, "r": 10, "t": 60, "b": 10},
|
||||||
|
)
|
||||||
|
|
||||||
|
return fig
|
||||||
|
|
||||||
|
|
||||||
|
def create_donut_chart(
|
||||||
|
data: list[dict[str, Any]],
|
||||||
|
name_column: str,
|
||||||
|
value_column: str,
|
||||||
|
title: str | None = None,
|
||||||
|
colors: list[str] | None = None,
|
||||||
|
hole_size: float = 0.4,
|
||||||
|
) -> go.Figure:
|
||||||
|
"""Create donut chart for percentage breakdowns.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
data: List of data records with name and value.
|
||||||
|
name_column: Column name for labels.
|
||||||
|
value_column: Column name for values.
|
||||||
|
title: Optional chart title.
|
||||||
|
colors: List of colors for segments.
|
||||||
|
hole_size: Size of center hole (0-1).
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Plotly Figure object.
|
||||||
|
"""
|
||||||
|
if not data:
|
||||||
|
return _create_empty_figure(title or "Distribution")
|
||||||
|
|
||||||
|
df = pd.DataFrame(data)
|
||||||
|
|
||||||
|
if colors is None:
|
||||||
|
colors = [
|
||||||
|
"#2196F3",
|
||||||
|
"#4CAF50",
|
||||||
|
"#FF9800",
|
||||||
|
"#E91E63",
|
||||||
|
"#9C27B0",
|
||||||
|
"#00BCD4",
|
||||||
|
"#FFC107",
|
||||||
|
"#795548",
|
||||||
|
]
|
||||||
|
|
||||||
|
fig = go.Figure(
|
||||||
|
go.Pie(
|
||||||
|
labels=df[name_column],
|
||||||
|
values=df[value_column],
|
||||||
|
hole=hole_size,
|
||||||
|
marker_colors=colors[: len(df)],
|
||||||
|
textinfo="percent+label",
|
||||||
|
textposition="outside",
|
||||||
|
hovertemplate="%{label}<br>%{value:,} (%{percent})<extra></extra>",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
fig.update_layout(
|
||||||
|
title=title,
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
showlegend=False,
|
||||||
|
margin={"l": 10, "r": 10, "t": 60, "b": 10},
|
||||||
|
)
|
||||||
|
|
||||||
|
return fig
|
||||||
|
|
||||||
|
|
||||||
|
def create_income_distribution(
|
||||||
|
data: list[dict[str, Any]],
|
||||||
|
bracket_column: str,
|
||||||
|
count_column: str,
|
||||||
|
title: str | None = None,
|
||||||
|
color: str = "#4CAF50",
|
||||||
|
) -> go.Figure:
|
||||||
|
"""Create histogram-style bar chart for income distribution.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
data: List of data records with income brackets and counts.
|
||||||
|
bracket_column: Column name for income brackets.
|
||||||
|
count_column: Column name for household counts.
|
||||||
|
title: Optional chart title.
|
||||||
|
color: Bar color.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Plotly Figure object.
|
||||||
|
"""
|
||||||
|
if not data:
|
||||||
|
return _create_empty_figure(title or "Income Distribution")
|
||||||
|
|
||||||
|
df = pd.DataFrame(data)
|
||||||
|
|
||||||
|
fig = go.Figure(
|
||||||
|
go.Bar(
|
||||||
|
x=df[bracket_column],
|
||||||
|
y=df[count_column],
|
||||||
|
marker_color=color,
|
||||||
|
text=df[count_column].apply(lambda x: f"{x:,}"),
|
||||||
|
textposition="outside",
|
||||||
|
hovertemplate="%{x}<br>Households: %{y:,}<extra></extra>",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
fig.update_layout(
|
||||||
|
title=title,
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
plot_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
xaxis={
|
||||||
|
"title": "Income Bracket",
|
||||||
|
"gridcolor": "rgba(128,128,128,0.2)",
|
||||||
|
"tickangle": -45,
|
||||||
|
},
|
||||||
|
yaxis={
|
||||||
|
"title": "Households",
|
||||||
|
"gridcolor": "rgba(128,128,128,0.2)",
|
||||||
|
},
|
||||||
|
margin={"l": 10, "r": 10, "t": 60, "b": 80},
|
||||||
|
)
|
||||||
|
|
||||||
|
return fig
|
||||||
|
|
||||||
|
|
||||||
|
def _create_empty_figure(title: str) -> go.Figure:
|
||||||
|
"""Create an empty figure with a message."""
|
||||||
|
fig = go.Figure()
|
||||||
|
fig.add_annotation(
|
||||||
|
text="No data available",
|
||||||
|
xref="paper",
|
||||||
|
yref="paper",
|
||||||
|
x=0.5,
|
||||||
|
y=0.5,
|
||||||
|
showarrow=False,
|
||||||
|
font={"size": 14, "color": "#888888"},
|
||||||
|
)
|
||||||
|
fig.update_layout(
|
||||||
|
title=title,
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
plot_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
xaxis={"visible": False},
|
||||||
|
yaxis={"visible": False},
|
||||||
|
)
|
||||||
|
return fig
|
||||||
166
portfolio_app/figures/radar.py
Normal file
166
portfolio_app/figures/radar.py
Normal file
@@ -0,0 +1,166 @@
|
|||||||
|
"""Radar/spider chart figure factory for multi-metric comparison."""
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import plotly.graph_objects as go
|
||||||
|
|
||||||
|
|
||||||
|
def create_radar_figure(
|
||||||
|
data: list[dict[str, Any]],
|
||||||
|
metrics: list[str],
|
||||||
|
name_column: str | None = None,
|
||||||
|
title: str | None = None,
|
||||||
|
fill: bool = True,
|
||||||
|
colors: list[str] | None = None,
|
||||||
|
) -> go.Figure:
|
||||||
|
"""Create radar/spider chart for multi-axis comparison.
|
||||||
|
|
||||||
|
Each record in data represents one entity (e.g., a neighbourhood)
|
||||||
|
with values for each metric that will be plotted on a separate axis.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
data: List of data records, each with values for the metrics.
|
||||||
|
metrics: List of metric column names to display on radar axes.
|
||||||
|
name_column: Column name for entity labels.
|
||||||
|
title: Optional chart title.
|
||||||
|
fill: Whether to fill the radar polygons.
|
||||||
|
colors: List of colors for each data series.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Plotly Figure object.
|
||||||
|
"""
|
||||||
|
if not data or not metrics:
|
||||||
|
return _create_empty_figure(title or "Radar Chart")
|
||||||
|
|
||||||
|
# Default colors
|
||||||
|
if colors is None:
|
||||||
|
colors = [
|
||||||
|
"#2196F3",
|
||||||
|
"#4CAF50",
|
||||||
|
"#FF9800",
|
||||||
|
"#E91E63",
|
||||||
|
"#9C27B0",
|
||||||
|
"#00BCD4",
|
||||||
|
]
|
||||||
|
|
||||||
|
fig = go.Figure()
|
||||||
|
|
||||||
|
# Format axis labels
|
||||||
|
axis_labels = [m.replace("_", " ").title() for m in metrics]
|
||||||
|
|
||||||
|
for i, record in enumerate(data):
|
||||||
|
values = [record.get(m, 0) or 0 for m in metrics]
|
||||||
|
# Close the radar polygon
|
||||||
|
values_closed = values + [values[0]]
|
||||||
|
labels_closed = axis_labels + [axis_labels[0]]
|
||||||
|
|
||||||
|
name = (
|
||||||
|
record.get(name_column, f"Series {i + 1}")
|
||||||
|
if name_column
|
||||||
|
else f"Series {i + 1}"
|
||||||
|
)
|
||||||
|
color = colors[i % len(colors)]
|
||||||
|
|
||||||
|
fig.add_trace(
|
||||||
|
go.Scatterpolar(
|
||||||
|
r=values_closed,
|
||||||
|
theta=labels_closed,
|
||||||
|
name=name,
|
||||||
|
line={"color": color, "width": 2},
|
||||||
|
fill="toself" if fill else None,
|
||||||
|
fillcolor=f"rgba{_hex_to_rgba(color, 0.2)}" if fill else None,
|
||||||
|
hovertemplate="%{theta}: %{r:.1f}<extra></extra>",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
fig.update_layout(
|
||||||
|
title=title,
|
||||||
|
polar={
|
||||||
|
"radialaxis": {
|
||||||
|
"visible": True,
|
||||||
|
"gridcolor": "rgba(128,128,128,0.3)",
|
||||||
|
"linecolor": "rgba(128,128,128,0.3)",
|
||||||
|
"tickfont": {"color": "#c9c9c9"},
|
||||||
|
},
|
||||||
|
"angularaxis": {
|
||||||
|
"gridcolor": "rgba(128,128,128,0.3)",
|
||||||
|
"linecolor": "rgba(128,128,128,0.3)",
|
||||||
|
"tickfont": {"color": "#c9c9c9"},
|
||||||
|
},
|
||||||
|
"bgcolor": "rgba(0,0,0,0)",
|
||||||
|
},
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
showlegend=len(data) > 1,
|
||||||
|
legend={"orientation": "h", "yanchor": "bottom", "y": -0.2},
|
||||||
|
margin={"l": 40, "r": 40, "t": 60, "b": 40},
|
||||||
|
)
|
||||||
|
|
||||||
|
return fig
|
||||||
|
|
||||||
|
|
||||||
|
def create_comparison_radar(
|
||||||
|
selected_data: dict[str, Any],
|
||||||
|
average_data: dict[str, Any],
|
||||||
|
metrics: list[str],
|
||||||
|
selected_name: str = "Selected",
|
||||||
|
average_name: str = "City Average",
|
||||||
|
title: str | None = None,
|
||||||
|
) -> go.Figure:
|
||||||
|
"""Create radar chart comparing a selection to city average.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
selected_data: Data for the selected entity.
|
||||||
|
average_data: Data for the city average.
|
||||||
|
metrics: List of metric column names.
|
||||||
|
selected_name: Label for selected entity.
|
||||||
|
average_name: Label for average.
|
||||||
|
title: Optional chart title.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Plotly Figure object.
|
||||||
|
"""
|
||||||
|
if not selected_data or not average_data:
|
||||||
|
return _create_empty_figure(title or "Comparison")
|
||||||
|
|
||||||
|
data = [
|
||||||
|
{**selected_data, "__name__": selected_name},
|
||||||
|
{**average_data, "__name__": average_name},
|
||||||
|
]
|
||||||
|
|
||||||
|
return create_radar_figure(
|
||||||
|
data=data,
|
||||||
|
metrics=metrics,
|
||||||
|
name_column="__name__",
|
||||||
|
title=title,
|
||||||
|
colors=["#4CAF50", "#9E9E9E"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _hex_to_rgba(hex_color: str, alpha: float) -> tuple[int, int, int, float]:
|
||||||
|
"""Convert hex color to RGBA tuple."""
|
||||||
|
hex_color = hex_color.lstrip("#")
|
||||||
|
r = int(hex_color[0:2], 16)
|
||||||
|
g = int(hex_color[2:4], 16)
|
||||||
|
b = int(hex_color[4:6], 16)
|
||||||
|
return (r, g, b, alpha)
|
||||||
|
|
||||||
|
|
||||||
|
def _create_empty_figure(title: str) -> go.Figure:
|
||||||
|
"""Create an empty figure with a message."""
|
||||||
|
fig = go.Figure()
|
||||||
|
fig.add_annotation(
|
||||||
|
text="No data available",
|
||||||
|
xref="paper",
|
||||||
|
yref="paper",
|
||||||
|
x=0.5,
|
||||||
|
y=0.5,
|
||||||
|
showarrow=False,
|
||||||
|
font={"size": 14, "color": "#888888"},
|
||||||
|
)
|
||||||
|
fig.update_layout(
|
||||||
|
title=title,
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
)
|
||||||
|
return fig
|
||||||
184
portfolio_app/figures/scatter.py
Normal file
184
portfolio_app/figures/scatter.py
Normal file
@@ -0,0 +1,184 @@
|
|||||||
|
"""Scatter plot figure factory for correlation views."""
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pandas as pd
|
||||||
|
import plotly.express as px
|
||||||
|
import plotly.graph_objects as go
|
||||||
|
|
||||||
|
|
||||||
|
def create_scatter_figure(
|
||||||
|
data: list[dict[str, Any]],
|
||||||
|
x_column: str,
|
||||||
|
y_column: str,
|
||||||
|
name_column: str | None = None,
|
||||||
|
size_column: str | None = None,
|
||||||
|
color_column: str | None = None,
|
||||||
|
title: str | None = None,
|
||||||
|
x_title: str | None = None,
|
||||||
|
y_title: str | None = None,
|
||||||
|
trendline: bool = False,
|
||||||
|
color_scale: str = "Blues",
|
||||||
|
) -> go.Figure:
|
||||||
|
"""Create scatter plot for correlation visualization.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
data: List of data records.
|
||||||
|
x_column: Column name for x-axis values.
|
||||||
|
y_column: Column name for y-axis values.
|
||||||
|
name_column: Column name for point labels (hover).
|
||||||
|
size_column: Column name for point sizes.
|
||||||
|
color_column: Column name for color encoding.
|
||||||
|
title: Optional chart title.
|
||||||
|
x_title: X-axis title.
|
||||||
|
y_title: Y-axis title.
|
||||||
|
trendline: Whether to add OLS trendline.
|
||||||
|
color_scale: Plotly color scale for continuous colors.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Plotly Figure object.
|
||||||
|
"""
|
||||||
|
if not data:
|
||||||
|
return _create_empty_figure(title or "Scatter Plot")
|
||||||
|
|
||||||
|
df = pd.DataFrame(data)
|
||||||
|
|
||||||
|
# Build hover_data
|
||||||
|
hover_data = {}
|
||||||
|
if name_column and name_column in df.columns:
|
||||||
|
hover_data[name_column] = True
|
||||||
|
|
||||||
|
# Create scatter plot
|
||||||
|
fig = px.scatter(
|
||||||
|
df,
|
||||||
|
x=x_column,
|
||||||
|
y=y_column,
|
||||||
|
size=size_column if size_column and size_column in df.columns else None,
|
||||||
|
color=color_column if color_column and color_column in df.columns else None,
|
||||||
|
color_continuous_scale=color_scale,
|
||||||
|
hover_name=name_column,
|
||||||
|
trendline="ols" if trendline else None,
|
||||||
|
opacity=0.7,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Style the markers
|
||||||
|
fig.update_traces(
|
||||||
|
marker={
|
||||||
|
"line": {"width": 1, "color": "rgba(255,255,255,0.3)"},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
# Trendline styling
|
||||||
|
if trendline:
|
||||||
|
fig.update_traces(
|
||||||
|
selector={"mode": "lines"},
|
||||||
|
line={"color": "#FF9800", "dash": "dash", "width": 2},
|
||||||
|
)
|
||||||
|
|
||||||
|
fig.update_layout(
|
||||||
|
title=title,
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
plot_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
xaxis={
|
||||||
|
"gridcolor": "rgba(128,128,128,0.2)",
|
||||||
|
"title": x_title or x_column.replace("_", " ").title(),
|
||||||
|
"zeroline": False,
|
||||||
|
},
|
||||||
|
yaxis={
|
||||||
|
"gridcolor": "rgba(128,128,128,0.2)",
|
||||||
|
"title": y_title or y_column.replace("_", " ").title(),
|
||||||
|
"zeroline": False,
|
||||||
|
},
|
||||||
|
margin={"l": 10, "r": 10, "t": 40, "b": 10},
|
||||||
|
showlegend=color_column is not None,
|
||||||
|
)
|
||||||
|
|
||||||
|
return fig
|
||||||
|
|
||||||
|
|
||||||
|
def create_bubble_chart(
|
||||||
|
data: list[dict[str, Any]],
|
||||||
|
x_column: str,
|
||||||
|
y_column: str,
|
||||||
|
size_column: str,
|
||||||
|
name_column: str | None = None,
|
||||||
|
color_column: str | None = None,
|
||||||
|
title: str | None = None,
|
||||||
|
x_title: str | None = None,
|
||||||
|
y_title: str | None = None,
|
||||||
|
size_max: int = 50,
|
||||||
|
) -> go.Figure:
|
||||||
|
"""Create bubble chart with sized markers.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
data: List of data records.
|
||||||
|
x_column: Column name for x-axis values.
|
||||||
|
y_column: Column name for y-axis values.
|
||||||
|
size_column: Column name for bubble sizes.
|
||||||
|
name_column: Column name for labels.
|
||||||
|
color_column: Column name for colors.
|
||||||
|
title: Optional chart title.
|
||||||
|
x_title: X-axis title.
|
||||||
|
y_title: Y-axis title.
|
||||||
|
size_max: Maximum marker size in pixels.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Plotly Figure object.
|
||||||
|
"""
|
||||||
|
if not data:
|
||||||
|
return _create_empty_figure(title or "Bubble Chart")
|
||||||
|
|
||||||
|
df = pd.DataFrame(data)
|
||||||
|
|
||||||
|
fig = px.scatter(
|
||||||
|
df,
|
||||||
|
x=x_column,
|
||||||
|
y=y_column,
|
||||||
|
size=size_column,
|
||||||
|
color=color_column,
|
||||||
|
hover_name=name_column,
|
||||||
|
size_max=size_max,
|
||||||
|
opacity=0.7,
|
||||||
|
)
|
||||||
|
|
||||||
|
fig.update_layout(
|
||||||
|
title=title,
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
plot_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
xaxis={
|
||||||
|
"gridcolor": "rgba(128,128,128,0.2)",
|
||||||
|
"title": x_title or x_column.replace("_", " ").title(),
|
||||||
|
},
|
||||||
|
yaxis={
|
||||||
|
"gridcolor": "rgba(128,128,128,0.2)",
|
||||||
|
"title": y_title or y_column.replace("_", " ").title(),
|
||||||
|
},
|
||||||
|
margin={"l": 10, "r": 10, "t": 40, "b": 10},
|
||||||
|
)
|
||||||
|
|
||||||
|
return fig
|
||||||
|
|
||||||
|
|
||||||
|
def _create_empty_figure(title: str) -> go.Figure:
|
||||||
|
"""Create an empty figure with a message."""
|
||||||
|
fig = go.Figure()
|
||||||
|
fig.add_annotation(
|
||||||
|
text="No data available",
|
||||||
|
xref="paper",
|
||||||
|
yref="paper",
|
||||||
|
x=0.5,
|
||||||
|
y=0.5,
|
||||||
|
showarrow=False,
|
||||||
|
font={"size": 14, "color": "#888888"},
|
||||||
|
)
|
||||||
|
fig.update_layout(
|
||||||
|
title=title,
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
plot_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
xaxis={"visible": False},
|
||||||
|
yaxis={"visible": False},
|
||||||
|
)
|
||||||
|
return fig
|
||||||
File diff suppressed because it is too large
Load Diff
385
portfolio_app/pages/toronto/callbacks/chart_callbacks.py
Normal file
385
portfolio_app/pages/toronto/callbacks/chart_callbacks.py
Normal file
@@ -0,0 +1,385 @@
|
|||||||
|
"""Chart callbacks for supporting visualizations."""
|
||||||
|
# mypy: disable-error-code="misc,no-untyped-def,arg-type"
|
||||||
|
|
||||||
|
import plotly.graph_objects as go
|
||||||
|
from dash import Input, Output, callback
|
||||||
|
|
||||||
|
from portfolio_app.figures import (
|
||||||
|
create_donut_chart,
|
||||||
|
create_horizontal_bar,
|
||||||
|
create_radar_figure,
|
||||||
|
create_scatter_figure,
|
||||||
|
)
|
||||||
|
from portfolio_app.toronto.services import (
|
||||||
|
get_amenities_data,
|
||||||
|
get_city_averages,
|
||||||
|
get_demographics_data,
|
||||||
|
get_housing_data,
|
||||||
|
get_neighbourhood_details,
|
||||||
|
get_safety_data,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("overview-scatter-chart", "figure"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_overview_scatter(year: str) -> go.Figure:
|
||||||
|
"""Update income vs safety scatter plot."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
df = get_demographics_data(year_int)
|
||||||
|
safety_df = get_safety_data(year_int)
|
||||||
|
|
||||||
|
if df.empty or safety_df.empty:
|
||||||
|
return _empty_chart("No data available")
|
||||||
|
|
||||||
|
# Merge demographics with safety
|
||||||
|
merged = df.merge(
|
||||||
|
safety_df[["neighbourhood_id", "total_crime_rate"]],
|
||||||
|
on="neighbourhood_id",
|
||||||
|
how="left",
|
||||||
|
)
|
||||||
|
|
||||||
|
# Compute safety score (inverse of crime rate)
|
||||||
|
if "total_crime_rate" in merged.columns:
|
||||||
|
max_crime = merged["total_crime_rate"].max()
|
||||||
|
merged["safety_score"] = 100 - (merged["total_crime_rate"] / max_crime * 100)
|
||||||
|
|
||||||
|
data = merged.to_dict("records")
|
||||||
|
|
||||||
|
return create_scatter_figure(
|
||||||
|
data=data,
|
||||||
|
x_column="median_household_income",
|
||||||
|
y_column="safety_score",
|
||||||
|
name_column="neighbourhood_name",
|
||||||
|
size_column="population",
|
||||||
|
title="Income vs Safety",
|
||||||
|
x_title="Median Household Income ($)",
|
||||||
|
y_title="Safety Score",
|
||||||
|
trendline=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("housing-trend-chart", "figure"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
Input("toronto-selected-neighbourhood", "data"),
|
||||||
|
)
|
||||||
|
def update_housing_trend(year: str, neighbourhood_id: int | None) -> go.Figure:
|
||||||
|
"""Update housing rent trend chart."""
|
||||||
|
# For now, show city averages as we don't have multi-year data
|
||||||
|
# This would be a time series if we had historical data
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
averages = get_city_averages(year_int)
|
||||||
|
|
||||||
|
if not averages:
|
||||||
|
return _empty_chart("No trend data available")
|
||||||
|
|
||||||
|
# Placeholder for trend data - would be historical
|
||||||
|
data = [
|
||||||
|
{"year": "2019", "avg_rent": averages.get("avg_rent_2bed", 2000) * 0.85},
|
||||||
|
{"year": "2020", "avg_rent": averages.get("avg_rent_2bed", 2000) * 0.88},
|
||||||
|
{"year": "2021", "avg_rent": averages.get("avg_rent_2bed", 2000) * 0.92},
|
||||||
|
{"year": "2022", "avg_rent": averages.get("avg_rent_2bed", 2000) * 0.96},
|
||||||
|
{"year": "2023", "avg_rent": averages.get("avg_rent_2bed", 2000)},
|
||||||
|
]
|
||||||
|
|
||||||
|
fig = go.Figure()
|
||||||
|
fig.add_trace(
|
||||||
|
go.Scatter(
|
||||||
|
x=[d["year"] for d in data],
|
||||||
|
y=[d["avg_rent"] for d in data],
|
||||||
|
mode="lines+markers",
|
||||||
|
line={"color": "#2196F3", "width": 2},
|
||||||
|
marker={"size": 8},
|
||||||
|
name="City Average",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
fig.update_layout(
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
plot_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
xaxis={"gridcolor": "rgba(128,128,128,0.2)"},
|
||||||
|
yaxis={"gridcolor": "rgba(128,128,128,0.2)", "title": "Avg Rent (2BR)"},
|
||||||
|
showlegend=False,
|
||||||
|
margin={"l": 40, "r": 10, "t": 10, "b": 30},
|
||||||
|
)
|
||||||
|
|
||||||
|
return fig
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("housing-types-chart", "figure"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_housing_types(year: str) -> go.Figure:
|
||||||
|
"""Update dwelling types breakdown chart."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
df = get_housing_data(year_int)
|
||||||
|
|
||||||
|
if df.empty:
|
||||||
|
return _empty_chart("No data available")
|
||||||
|
|
||||||
|
# Aggregate tenure types across city
|
||||||
|
owner_pct = df["pct_owner_occupied"].mean()
|
||||||
|
renter_pct = df["pct_renter_occupied"].mean()
|
||||||
|
|
||||||
|
data = [
|
||||||
|
{"type": "Owner Occupied", "percentage": owner_pct},
|
||||||
|
{"type": "Renter Occupied", "percentage": renter_pct},
|
||||||
|
]
|
||||||
|
|
||||||
|
return create_donut_chart(
|
||||||
|
data=data,
|
||||||
|
name_column="type",
|
||||||
|
value_column="percentage",
|
||||||
|
colors=["#4CAF50", "#2196F3"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("safety-trend-chart", "figure"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_safety_trend(year: str) -> go.Figure:
|
||||||
|
"""Update crime trend chart."""
|
||||||
|
# Placeholder for trend - would need historical data
|
||||||
|
data = [
|
||||||
|
{"year": "2019", "crime_rate": 4500},
|
||||||
|
{"year": "2020", "crime_rate": 4200},
|
||||||
|
{"year": "2021", "crime_rate": 4100},
|
||||||
|
{"year": "2022", "crime_rate": 4300},
|
||||||
|
{"year": "2023", "crime_rate": 4250},
|
||||||
|
]
|
||||||
|
|
||||||
|
fig = go.Figure()
|
||||||
|
fig.add_trace(
|
||||||
|
go.Scatter(
|
||||||
|
x=[d["year"] for d in data],
|
||||||
|
y=[d["crime_rate"] for d in data],
|
||||||
|
mode="lines+markers",
|
||||||
|
line={"color": "#FF5722", "width": 2},
|
||||||
|
marker={"size": 8},
|
||||||
|
fill="tozeroy",
|
||||||
|
fillcolor="rgba(255,87,34,0.1)",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
fig.update_layout(
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
plot_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
xaxis={"gridcolor": "rgba(128,128,128,0.2)"},
|
||||||
|
yaxis={"gridcolor": "rgba(128,128,128,0.2)", "title": "Crime Rate per 100K"},
|
||||||
|
showlegend=False,
|
||||||
|
margin={"l": 40, "r": 10, "t": 10, "b": 30},
|
||||||
|
)
|
||||||
|
|
||||||
|
return fig
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("safety-types-chart", "figure"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_safety_types(year: str) -> go.Figure:
|
||||||
|
"""Update crime by category chart."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
df = get_safety_data(year_int)
|
||||||
|
|
||||||
|
if df.empty:
|
||||||
|
return _empty_chart("No data available")
|
||||||
|
|
||||||
|
# Aggregate crime types across city
|
||||||
|
violent = df["violent_crimes"].sum() if "violent_crimes" in df.columns else 0
|
||||||
|
property_crimes = (
|
||||||
|
df["property_crimes"].sum() if "property_crimes" in df.columns else 0
|
||||||
|
)
|
||||||
|
theft = df["theft_crimes"].sum() if "theft_crimes" in df.columns else 0
|
||||||
|
other = (
|
||||||
|
df["total_crimes"].sum() - violent - property_crimes - theft
|
||||||
|
if "total_crimes" in df.columns
|
||||||
|
else 0
|
||||||
|
)
|
||||||
|
|
||||||
|
data = [
|
||||||
|
{"category": "Violent", "count": int(violent)},
|
||||||
|
{"category": "Property", "count": int(property_crimes)},
|
||||||
|
{"category": "Theft", "count": int(theft)},
|
||||||
|
{"category": "Other", "count": int(max(0, other))},
|
||||||
|
]
|
||||||
|
|
||||||
|
return create_horizontal_bar(
|
||||||
|
data=data,
|
||||||
|
name_column="category",
|
||||||
|
value_column="count",
|
||||||
|
color="#FF5722",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("demographics-age-chart", "figure"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_demographics_age(year: str) -> go.Figure:
|
||||||
|
"""Update age distribution chart."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
df = get_demographics_data(year_int)
|
||||||
|
|
||||||
|
if df.empty:
|
||||||
|
return _empty_chart("No data available")
|
||||||
|
|
||||||
|
# Calculate average age distribution
|
||||||
|
under_18 = df["pct_under_18"].mean() if "pct_under_18" in df.columns else 20
|
||||||
|
age_18_64 = df["pct_18_to_64"].mean() if "pct_18_to_64" in df.columns else 65
|
||||||
|
over_65 = df["pct_65_plus"].mean() if "pct_65_plus" in df.columns else 15
|
||||||
|
|
||||||
|
data = [
|
||||||
|
{"age_group": "Under 18", "percentage": under_18},
|
||||||
|
{"age_group": "18-64", "percentage": age_18_64},
|
||||||
|
{"age_group": "65+", "percentage": over_65},
|
||||||
|
]
|
||||||
|
|
||||||
|
return create_donut_chart(
|
||||||
|
data=data,
|
||||||
|
name_column="age_group",
|
||||||
|
value_column="percentage",
|
||||||
|
colors=["#9C27B0", "#673AB7", "#3F51B5"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("demographics-income-chart", "figure"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_demographics_income(year: str) -> go.Figure:
|
||||||
|
"""Update income distribution chart."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
df = get_demographics_data(year_int)
|
||||||
|
|
||||||
|
if df.empty:
|
||||||
|
return _empty_chart("No data available")
|
||||||
|
|
||||||
|
# Create income quintile distribution
|
||||||
|
if "income_quintile" in df.columns:
|
||||||
|
quintile_counts = df["income_quintile"].value_counts().sort_index()
|
||||||
|
data = [
|
||||||
|
{"bracket": f"Q{q}", "count": int(count)}
|
||||||
|
for q, count in quintile_counts.items()
|
||||||
|
]
|
||||||
|
else:
|
||||||
|
# Fallback to placeholder
|
||||||
|
data = [
|
||||||
|
{"bracket": "Q1 (Low)", "count": 32},
|
||||||
|
{"bracket": "Q2", "count": 32},
|
||||||
|
{"bracket": "Q3 (Mid)", "count": 32},
|
||||||
|
{"bracket": "Q4", "count": 31},
|
||||||
|
{"bracket": "Q5 (High)", "count": 31},
|
||||||
|
]
|
||||||
|
|
||||||
|
return create_horizontal_bar(
|
||||||
|
data=data,
|
||||||
|
name_column="bracket",
|
||||||
|
value_column="count",
|
||||||
|
color="#4CAF50",
|
||||||
|
sort=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("amenities-breakdown-chart", "figure"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_amenities_breakdown(year: str) -> go.Figure:
|
||||||
|
"""Update amenity breakdown chart."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
df = get_amenities_data(year_int)
|
||||||
|
|
||||||
|
if df.empty:
|
||||||
|
return _empty_chart("No data available")
|
||||||
|
|
||||||
|
# Aggregate amenity counts
|
||||||
|
parks = df["park_count"].sum() if "park_count" in df.columns else 0
|
||||||
|
schools = df["school_count"].sum() if "school_count" in df.columns else 0
|
||||||
|
childcare = df["childcare_count"].sum() if "childcare_count" in df.columns else 0
|
||||||
|
|
||||||
|
data = [
|
||||||
|
{"type": "Parks", "count": int(parks)},
|
||||||
|
{"type": "Schools", "count": int(schools)},
|
||||||
|
{"type": "Childcare", "count": int(childcare)},
|
||||||
|
]
|
||||||
|
|
||||||
|
return create_horizontal_bar(
|
||||||
|
data=data,
|
||||||
|
name_column="type",
|
||||||
|
value_column="count",
|
||||||
|
color="#4CAF50",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("amenities-radar-chart", "figure"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
Input("toronto-selected-neighbourhood", "data"),
|
||||||
|
)
|
||||||
|
def update_amenities_radar(year: str, neighbourhood_id: int | None) -> go.Figure:
|
||||||
|
"""Update amenity comparison radar chart."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
|
||||||
|
# Get city averages
|
||||||
|
averages = get_city_averages(year_int)
|
||||||
|
|
||||||
|
city_data = {
|
||||||
|
"parks_per_1000": averages.get("avg_amenity_score", 50) / 100 * 10,
|
||||||
|
"schools_per_1000": averages.get("avg_amenity_score", 50) / 100 * 5,
|
||||||
|
"childcare_per_1000": averages.get("avg_amenity_score", 50) / 100 * 3,
|
||||||
|
"transit_access": 70,
|
||||||
|
}
|
||||||
|
|
||||||
|
data = [city_data]
|
||||||
|
|
||||||
|
# Add selected neighbourhood if available
|
||||||
|
if neighbourhood_id:
|
||||||
|
details = get_neighbourhood_details(neighbourhood_id, year_int)
|
||||||
|
if details:
|
||||||
|
selected_data = {
|
||||||
|
"parks_per_1000": details.get("park_count", 0) / 10,
|
||||||
|
"schools_per_1000": details.get("school_count", 0) / 5,
|
||||||
|
"childcare_per_1000": 3,
|
||||||
|
"transit_access": 70,
|
||||||
|
}
|
||||||
|
data.insert(0, selected_data)
|
||||||
|
|
||||||
|
return create_radar_figure(
|
||||||
|
data=data,
|
||||||
|
metrics=[
|
||||||
|
"parks_per_1000",
|
||||||
|
"schools_per_1000",
|
||||||
|
"childcare_per_1000",
|
||||||
|
"transit_access",
|
||||||
|
],
|
||||||
|
fill=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _empty_chart(message: str) -> go.Figure:
|
||||||
|
"""Create an empty chart with a message."""
|
||||||
|
fig = go.Figure()
|
||||||
|
fig.update_layout(
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
plot_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
xaxis={"visible": False},
|
||||||
|
yaxis={"visible": False},
|
||||||
|
)
|
||||||
|
fig.add_annotation(
|
||||||
|
text=message,
|
||||||
|
xref="paper",
|
||||||
|
yref="paper",
|
||||||
|
x=0.5,
|
||||||
|
y=0.5,
|
||||||
|
showarrow=False,
|
||||||
|
font={"size": 14, "color": "#888888"},
|
||||||
|
)
|
||||||
|
return fig
|
||||||
304
portfolio_app/pages/toronto/callbacks/map_callbacks.py
Normal file
304
portfolio_app/pages/toronto/callbacks/map_callbacks.py
Normal file
@@ -0,0 +1,304 @@
|
|||||||
|
"""Map callbacks for choropleth interactions."""
|
||||||
|
# mypy: disable-error-code="misc,no-untyped-def,arg-type,no-any-return"
|
||||||
|
|
||||||
|
import plotly.graph_objects as go
|
||||||
|
from dash import Input, Output, State, callback, no_update
|
||||||
|
|
||||||
|
from portfolio_app.figures import create_choropleth_figure, create_ranking_bar
|
||||||
|
from portfolio_app.toronto.services import (
|
||||||
|
get_amenities_data,
|
||||||
|
get_demographics_data,
|
||||||
|
get_housing_data,
|
||||||
|
get_neighbourhoods_geojson,
|
||||||
|
get_overview_data,
|
||||||
|
get_safety_data,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("overview-choropleth", "figure"),
|
||||||
|
Input("overview-metric-select", "value"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_overview_choropleth(metric: str, year: str) -> go.Figure:
|
||||||
|
"""Update the overview tab choropleth map."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
df = get_overview_data(year_int)
|
||||||
|
geojson = get_neighbourhoods_geojson(year_int)
|
||||||
|
|
||||||
|
if df.empty:
|
||||||
|
return _empty_map("No data available")
|
||||||
|
|
||||||
|
data = df.to_dict("records")
|
||||||
|
|
||||||
|
# Color scales based on metric
|
||||||
|
color_scale = {
|
||||||
|
"livability_score": "Viridis",
|
||||||
|
"safety_score": "Greens",
|
||||||
|
"affordability_score": "Blues",
|
||||||
|
"amenity_score": "Purples",
|
||||||
|
}.get(metric, "Viridis")
|
||||||
|
|
||||||
|
return create_choropleth_figure(
|
||||||
|
geojson=geojson,
|
||||||
|
data=data,
|
||||||
|
location_key="neighbourhood_id",
|
||||||
|
color_column=metric or "livability_score",
|
||||||
|
hover_data=["neighbourhood_name", "population"],
|
||||||
|
color_scale=color_scale,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("housing-choropleth", "figure"),
|
||||||
|
Input("housing-metric-select", "value"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_housing_choropleth(metric: str, year: str) -> go.Figure:
|
||||||
|
"""Update the housing tab choropleth map."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
df = get_housing_data(year_int)
|
||||||
|
geojson = get_neighbourhoods_geojson(year_int)
|
||||||
|
|
||||||
|
if df.empty:
|
||||||
|
return _empty_map("No housing data available")
|
||||||
|
|
||||||
|
data = df.to_dict("records")
|
||||||
|
|
||||||
|
color_scale = {
|
||||||
|
"affordability_index": "RdYlGn_r",
|
||||||
|
"avg_rent_2bed": "Oranges",
|
||||||
|
"rent_to_income_pct": "Reds",
|
||||||
|
"vacancy_rate": "Blues",
|
||||||
|
}.get(metric, "Oranges")
|
||||||
|
|
||||||
|
return create_choropleth_figure(
|
||||||
|
geojson=geojson,
|
||||||
|
data=data,
|
||||||
|
location_key="neighbourhood_id",
|
||||||
|
color_column=metric or "affordability_index",
|
||||||
|
hover_data=["neighbourhood_name", "avg_rent_2bed", "vacancy_rate"],
|
||||||
|
color_scale=color_scale,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("safety-choropleth", "figure"),
|
||||||
|
Input("safety-metric-select", "value"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_safety_choropleth(metric: str, year: str) -> go.Figure:
|
||||||
|
"""Update the safety tab choropleth map."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
df = get_safety_data(year_int)
|
||||||
|
geojson = get_neighbourhoods_geojson(year_int)
|
||||||
|
|
||||||
|
if df.empty:
|
||||||
|
return _empty_map("No safety data available")
|
||||||
|
|
||||||
|
data = df.to_dict("records")
|
||||||
|
|
||||||
|
return create_choropleth_figure(
|
||||||
|
geojson=geojson,
|
||||||
|
data=data,
|
||||||
|
location_key="neighbourhood_id",
|
||||||
|
color_column=metric or "total_crime_rate",
|
||||||
|
hover_data=["neighbourhood_name", "total_crimes"],
|
||||||
|
color_scale="Reds",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("demographics-choropleth", "figure"),
|
||||||
|
Input("demographics-metric-select", "value"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_demographics_choropleth(metric: str, year: str) -> go.Figure:
|
||||||
|
"""Update the demographics tab choropleth map."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
df = get_demographics_data(year_int)
|
||||||
|
geojson = get_neighbourhoods_geojson(year_int)
|
||||||
|
|
||||||
|
if df.empty:
|
||||||
|
return _empty_map("No demographics data available")
|
||||||
|
|
||||||
|
data = df.to_dict("records")
|
||||||
|
|
||||||
|
color_scale = {
|
||||||
|
"population": "YlOrBr",
|
||||||
|
"median_income": "Greens",
|
||||||
|
"median_age": "Blues",
|
||||||
|
"diversity_index": "Purples",
|
||||||
|
}.get(metric, "YlOrBr")
|
||||||
|
|
||||||
|
# Map frontend metric names to column names
|
||||||
|
column_map = {
|
||||||
|
"population": "population",
|
||||||
|
"median_income": "median_household_income",
|
||||||
|
"median_age": "median_age",
|
||||||
|
"diversity_index": "diversity_index",
|
||||||
|
}
|
||||||
|
column = column_map.get(metric, "population")
|
||||||
|
|
||||||
|
return create_choropleth_figure(
|
||||||
|
geojson=geojson,
|
||||||
|
data=data,
|
||||||
|
location_key="neighbourhood_id",
|
||||||
|
color_column=column,
|
||||||
|
hover_data=["neighbourhood_name"],
|
||||||
|
color_scale=color_scale,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("amenities-choropleth", "figure"),
|
||||||
|
Input("amenities-metric-select", "value"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_amenities_choropleth(metric: str, year: str) -> go.Figure:
|
||||||
|
"""Update the amenities tab choropleth map."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
df = get_amenities_data(year_int)
|
||||||
|
geojson = get_neighbourhoods_geojson(year_int)
|
||||||
|
|
||||||
|
if df.empty:
|
||||||
|
return _empty_map("No amenities data available")
|
||||||
|
|
||||||
|
data = df.to_dict("records")
|
||||||
|
|
||||||
|
# Map frontend metric names to column names
|
||||||
|
column_map = {
|
||||||
|
"amenity_score": "amenity_score",
|
||||||
|
"parks_per_capita": "parks_per_1000",
|
||||||
|
"schools_per_capita": "schools_per_1000",
|
||||||
|
"transit_score": "total_amenities_per_1000",
|
||||||
|
}
|
||||||
|
column = column_map.get(metric, "amenity_score")
|
||||||
|
|
||||||
|
return create_choropleth_figure(
|
||||||
|
geojson=geojson,
|
||||||
|
data=data,
|
||||||
|
location_key="neighbourhood_id",
|
||||||
|
color_column=column,
|
||||||
|
hover_data=["neighbourhood_name", "park_count", "school_count"],
|
||||||
|
color_scale="Greens",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("toronto-selected-neighbourhood", "data"),
|
||||||
|
Input("overview-choropleth", "clickData"),
|
||||||
|
Input("housing-choropleth", "clickData"),
|
||||||
|
Input("safety-choropleth", "clickData"),
|
||||||
|
Input("demographics-choropleth", "clickData"),
|
||||||
|
Input("amenities-choropleth", "clickData"),
|
||||||
|
State("toronto-tabs", "value"),
|
||||||
|
prevent_initial_call=True,
|
||||||
|
)
|
||||||
|
def handle_map_click(
|
||||||
|
overview_click,
|
||||||
|
housing_click,
|
||||||
|
safety_click,
|
||||||
|
demographics_click,
|
||||||
|
amenities_click,
|
||||||
|
active_tab: str,
|
||||||
|
) -> int | None:
|
||||||
|
"""Extract neighbourhood ID from map click."""
|
||||||
|
# Get the click data for the active tab
|
||||||
|
click_map = {
|
||||||
|
"overview": overview_click,
|
||||||
|
"housing": housing_click,
|
||||||
|
"safety": safety_click,
|
||||||
|
"demographics": demographics_click,
|
||||||
|
"amenities": amenities_click,
|
||||||
|
}
|
||||||
|
|
||||||
|
click_data = click_map.get(active_tab)
|
||||||
|
|
||||||
|
if not click_data:
|
||||||
|
return no_update
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Extract neighbourhood_id from click data
|
||||||
|
point = click_data["points"][0]
|
||||||
|
location = point.get("location") or point.get("customdata", [None])[0]
|
||||||
|
if location:
|
||||||
|
return int(location)
|
||||||
|
except (KeyError, IndexError, TypeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
return no_update
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("overview-rankings-chart", "figure"),
|
||||||
|
Input("overview-metric-select", "value"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_rankings_chart(metric: str, year: str) -> go.Figure:
|
||||||
|
"""Update the top/bottom rankings bar chart."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
df = get_overview_data(year_int)
|
||||||
|
|
||||||
|
if df.empty:
|
||||||
|
return _empty_chart("No data available")
|
||||||
|
|
||||||
|
# Use the selected metric for ranking
|
||||||
|
metric = metric or "livability_score"
|
||||||
|
data = df.to_dict("records")
|
||||||
|
|
||||||
|
return create_ranking_bar(
|
||||||
|
data=data,
|
||||||
|
name_column="neighbourhood_name",
|
||||||
|
value_column=metric,
|
||||||
|
title=f"Top & Bottom 10 by {metric.replace('_', ' ').title()}",
|
||||||
|
top_n=10,
|
||||||
|
bottom_n=10,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _empty_map(message: str) -> go.Figure:
|
||||||
|
"""Create an empty map with a message."""
|
||||||
|
fig = go.Figure()
|
||||||
|
fig.update_layout(
|
||||||
|
mapbox={
|
||||||
|
"style": "carto-darkmatter",
|
||||||
|
"center": {"lat": 43.7, "lon": -79.4},
|
||||||
|
"zoom": 9.5,
|
||||||
|
},
|
||||||
|
margin={"l": 0, "r": 0, "t": 0, "b": 0},
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
)
|
||||||
|
fig.add_annotation(
|
||||||
|
text=message,
|
||||||
|
xref="paper",
|
||||||
|
yref="paper",
|
||||||
|
x=0.5,
|
||||||
|
y=0.5,
|
||||||
|
showarrow=False,
|
||||||
|
font={"size": 14, "color": "#888888"},
|
||||||
|
)
|
||||||
|
return fig
|
||||||
|
|
||||||
|
|
||||||
|
def _empty_chart(message: str) -> go.Figure:
|
||||||
|
"""Create an empty chart with a message."""
|
||||||
|
fig = go.Figure()
|
||||||
|
fig.update_layout(
|
||||||
|
paper_bgcolor="rgba(0,0,0,0)",
|
||||||
|
plot_bgcolor="rgba(0,0,0,0)",
|
||||||
|
font_color="#c9c9c9",
|
||||||
|
xaxis={"visible": False},
|
||||||
|
yaxis={"visible": False},
|
||||||
|
)
|
||||||
|
fig.add_annotation(
|
||||||
|
text=message,
|
||||||
|
xref="paper",
|
||||||
|
yref="paper",
|
||||||
|
x=0.5,
|
||||||
|
y=0.5,
|
||||||
|
showarrow=False,
|
||||||
|
font={"size": 14, "color": "#888888"},
|
||||||
|
)
|
||||||
|
return fig
|
||||||
309
portfolio_app/pages/toronto/callbacks/selection_callbacks.py
Normal file
309
portfolio_app/pages/toronto/callbacks/selection_callbacks.py
Normal file
@@ -0,0 +1,309 @@
|
|||||||
|
"""Selection callbacks for dropdowns and neighbourhood details."""
|
||||||
|
# mypy: disable-error-code="misc,no-untyped-def,type-arg"
|
||||||
|
|
||||||
|
import dash_mantine_components as dmc
|
||||||
|
from dash import Input, Output, callback
|
||||||
|
|
||||||
|
from portfolio_app.toronto.services import (
|
||||||
|
get_city_averages,
|
||||||
|
get_neighbourhood_details,
|
||||||
|
get_neighbourhood_list,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("toronto-neighbourhood-select", "data"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def populate_neighbourhood_dropdown(year: str) -> list[dict]:
|
||||||
|
"""Populate the neighbourhood search dropdown."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
neighbourhoods = get_neighbourhood_list(year_int)
|
||||||
|
|
||||||
|
return [
|
||||||
|
{"value": str(n["neighbourhood_id"]), "label": n["neighbourhood_name"]}
|
||||||
|
for n in neighbourhoods
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("toronto-selected-neighbourhood", "data", allow_duplicate=True),
|
||||||
|
Input("toronto-neighbourhood-select", "value"),
|
||||||
|
prevent_initial_call=True,
|
||||||
|
)
|
||||||
|
def select_from_dropdown(value: str | None) -> int | None:
|
||||||
|
"""Update selected neighbourhood from dropdown."""
|
||||||
|
if value:
|
||||||
|
return int(value)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("toronto-compare-btn", "disabled"),
|
||||||
|
Input("toronto-selected-neighbourhood", "data"),
|
||||||
|
)
|
||||||
|
def toggle_compare_button(neighbourhood_id: int | None) -> bool:
|
||||||
|
"""Enable compare button when a neighbourhood is selected."""
|
||||||
|
return neighbourhood_id is None
|
||||||
|
|
||||||
|
|
||||||
|
# Overview tab KPIs
|
||||||
|
@callback(
|
||||||
|
Output("overview-city-avg", "children"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_overview_city_avg(year: str) -> str:
|
||||||
|
"""Update the city average livability score."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
averages = get_city_averages(year_int)
|
||||||
|
score = averages.get("avg_livability_score", 72)
|
||||||
|
return f"{score:.0f}" if score else "—"
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("overview-selected-name", "children"),
|
||||||
|
Output("overview-selected-scores", "children"),
|
||||||
|
Input("toronto-selected-neighbourhood", "data"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_overview_selected(neighbourhood_id: int | None, year: str):
|
||||||
|
"""Update the selected neighbourhood details in overview tab."""
|
||||||
|
if not neighbourhood_id:
|
||||||
|
return "Click map to select", [dmc.Text("—", c="dimmed")]
|
||||||
|
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
details = get_neighbourhood_details(neighbourhood_id, year_int)
|
||||||
|
|
||||||
|
if not details:
|
||||||
|
return "Unknown", [dmc.Text("No data", c="dimmed")]
|
||||||
|
|
||||||
|
name = details.get("neighbourhood_name", "Unknown")
|
||||||
|
scores = [
|
||||||
|
dmc.Group(
|
||||||
|
[
|
||||||
|
dmc.Text("Livability:", size="sm"),
|
||||||
|
dmc.Text(
|
||||||
|
f"{details.get('livability_score', 0):.0f}", size="sm", fw=700
|
||||||
|
),
|
||||||
|
],
|
||||||
|
justify="space-between",
|
||||||
|
),
|
||||||
|
dmc.Group(
|
||||||
|
[
|
||||||
|
dmc.Text("Safety:", size="sm"),
|
||||||
|
dmc.Text(f"{details.get('safety_score', 0):.0f}", size="sm", fw=700),
|
||||||
|
],
|
||||||
|
justify="space-between",
|
||||||
|
),
|
||||||
|
dmc.Group(
|
||||||
|
[
|
||||||
|
dmc.Text("Affordability:", size="sm"),
|
||||||
|
dmc.Text(
|
||||||
|
f"{details.get('affordability_score', 0):.0f}", size="sm", fw=700
|
||||||
|
),
|
||||||
|
],
|
||||||
|
justify="space-between",
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
||||||
|
return name, scores
|
||||||
|
|
||||||
|
|
||||||
|
# Housing tab KPIs
|
||||||
|
@callback(
|
||||||
|
Output("housing-city-rent", "children"),
|
||||||
|
Output("housing-rent-change", "children"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_housing_kpis(year: str):
|
||||||
|
"""Update housing tab KPI cards."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
averages = get_city_averages(year_int)
|
||||||
|
|
||||||
|
rent = averages.get("avg_rent_2bed", 2450)
|
||||||
|
rent_str = f"${rent:,.0f}" if rent else "—"
|
||||||
|
|
||||||
|
# Placeholder change - would come from historical data
|
||||||
|
change = "+4.2% YoY"
|
||||||
|
|
||||||
|
return rent_str, change
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("housing-selected-name", "children"),
|
||||||
|
Output("housing-selected-details", "children"),
|
||||||
|
Input("toronto-selected-neighbourhood", "data"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_housing_selected(neighbourhood_id: int | None, year: str):
|
||||||
|
"""Update selected neighbourhood details in housing tab."""
|
||||||
|
if not neighbourhood_id:
|
||||||
|
return "Click map to select", [dmc.Text("—", c="dimmed")]
|
||||||
|
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
details = get_neighbourhood_details(neighbourhood_id, year_int)
|
||||||
|
|
||||||
|
if not details:
|
||||||
|
return "Unknown", [dmc.Text("No data", c="dimmed")]
|
||||||
|
|
||||||
|
name = details.get("neighbourhood_name", "Unknown")
|
||||||
|
rent = details.get("avg_rent_2bed")
|
||||||
|
vacancy = details.get("vacancy_rate")
|
||||||
|
|
||||||
|
info = [
|
||||||
|
dmc.Text(f"2BR Rent: ${rent:,.0f}" if rent else "2BR Rent: —", size="sm"),
|
||||||
|
dmc.Text(f"Vacancy: {vacancy:.1f}%" if vacancy else "Vacancy: —", size="sm"),
|
||||||
|
]
|
||||||
|
|
||||||
|
return name, info
|
||||||
|
|
||||||
|
|
||||||
|
# Safety tab KPIs
|
||||||
|
@callback(
|
||||||
|
Output("safety-city-rate", "children"),
|
||||||
|
Output("safety-rate-change", "children"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_safety_kpis(year: str):
|
||||||
|
"""Update safety tab KPI cards."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
averages = get_city_averages(year_int)
|
||||||
|
|
||||||
|
rate = averages.get("avg_crime_rate", 4250)
|
||||||
|
rate_str = f"{rate:,.0f}" if rate else "—"
|
||||||
|
|
||||||
|
# Placeholder change
|
||||||
|
change = "-2.1% YoY"
|
||||||
|
|
||||||
|
return rate_str, change
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("safety-selected-name", "children"),
|
||||||
|
Output("safety-selected-details", "children"),
|
||||||
|
Input("toronto-selected-neighbourhood", "data"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_safety_selected(neighbourhood_id: int | None, year: str):
|
||||||
|
"""Update selected neighbourhood details in safety tab."""
|
||||||
|
if not neighbourhood_id:
|
||||||
|
return "Click map to select", [dmc.Text("—", c="dimmed")]
|
||||||
|
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
details = get_neighbourhood_details(neighbourhood_id, year_int)
|
||||||
|
|
||||||
|
if not details:
|
||||||
|
return "Unknown", [dmc.Text("No data", c="dimmed")]
|
||||||
|
|
||||||
|
name = details.get("neighbourhood_name", "Unknown")
|
||||||
|
crime_rate = details.get("crime_rate_per_100k")
|
||||||
|
|
||||||
|
info = [
|
||||||
|
dmc.Text(
|
||||||
|
f"Crime Rate: {crime_rate:,.0f}/100K" if crime_rate else "Crime Rate: —",
|
||||||
|
size="sm",
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
||||||
|
return name, info
|
||||||
|
|
||||||
|
|
||||||
|
# Demographics tab KPIs
|
||||||
|
@callback(
|
||||||
|
Output("demographics-city-pop", "children"),
|
||||||
|
Output("demographics-pop-change", "children"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_demographics_kpis(year: str):
|
||||||
|
"""Update demographics tab KPI cards."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
averages = get_city_averages(year_int)
|
||||||
|
|
||||||
|
pop = averages.get("total_population", 2790000)
|
||||||
|
if pop and pop >= 1000000:
|
||||||
|
pop_str = f"{pop / 1000000:.2f}M"
|
||||||
|
elif pop:
|
||||||
|
pop_str = f"{pop:,.0f}"
|
||||||
|
else:
|
||||||
|
pop_str = "—"
|
||||||
|
|
||||||
|
change = "+2.3% since 2016"
|
||||||
|
|
||||||
|
return pop_str, change
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("demographics-selected-name", "children"),
|
||||||
|
Output("demographics-selected-details", "children"),
|
||||||
|
Input("toronto-selected-neighbourhood", "data"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_demographics_selected(neighbourhood_id: int | None, year: str):
|
||||||
|
"""Update selected neighbourhood details in demographics tab."""
|
||||||
|
if not neighbourhood_id:
|
||||||
|
return "Click map to select", [dmc.Text("—", c="dimmed")]
|
||||||
|
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
details = get_neighbourhood_details(neighbourhood_id, year_int)
|
||||||
|
|
||||||
|
if not details:
|
||||||
|
return "Unknown", [dmc.Text("No data", c="dimmed")]
|
||||||
|
|
||||||
|
name = details.get("neighbourhood_name", "Unknown")
|
||||||
|
pop = details.get("population")
|
||||||
|
income = details.get("median_household_income")
|
||||||
|
|
||||||
|
info = [
|
||||||
|
dmc.Text(f"Population: {pop:,}" if pop else "Population: —", size="sm"),
|
||||||
|
dmc.Text(
|
||||||
|
f"Median Income: ${income:,.0f}" if income else "Median Income: —",
|
||||||
|
size="sm",
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
||||||
|
return name, info
|
||||||
|
|
||||||
|
|
||||||
|
# Amenities tab KPIs
|
||||||
|
@callback(
|
||||||
|
Output("amenities-city-score", "children"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_amenities_kpis(year: str) -> str:
|
||||||
|
"""Update amenities tab KPI cards."""
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
averages = get_city_averages(year_int)
|
||||||
|
|
||||||
|
score = averages.get("avg_amenity_score", 68)
|
||||||
|
return f"{score:.0f}" if score else "—"
|
||||||
|
|
||||||
|
|
||||||
|
@callback(
|
||||||
|
Output("amenities-selected-name", "children"),
|
||||||
|
Output("amenities-selected-details", "children"),
|
||||||
|
Input("toronto-selected-neighbourhood", "data"),
|
||||||
|
Input("toronto-year-select", "value"),
|
||||||
|
)
|
||||||
|
def update_amenities_selected(neighbourhood_id: int | None, year: str):
|
||||||
|
"""Update selected neighbourhood details in amenities tab."""
|
||||||
|
if not neighbourhood_id:
|
||||||
|
return "Click map to select", [dmc.Text("—", c="dimmed")]
|
||||||
|
|
||||||
|
year_int = int(year) if year else 2021
|
||||||
|
details = get_neighbourhood_details(neighbourhood_id, year_int)
|
||||||
|
|
||||||
|
if not details:
|
||||||
|
return "Unknown", [dmc.Text("No data", c="dimmed")]
|
||||||
|
|
||||||
|
name = details.get("neighbourhood_name", "Unknown")
|
||||||
|
parks = details.get("park_count")
|
||||||
|
schools = details.get("school_count")
|
||||||
|
|
||||||
|
info = [
|
||||||
|
dmc.Text(f"Parks: {parks}" if parks is not None else "Parks: —", size="sm"),
|
||||||
|
dmc.Text(
|
||||||
|
f"Schools: {schools}" if schools is not None else "Schools: —", size="sm"
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
||||||
|
return name, info
|
||||||
@@ -1,62 +1,56 @@
|
|||||||
"""Toronto Housing Dashboard page."""
|
"""Toronto Neighbourhood Dashboard page.
|
||||||
|
|
||||||
|
Displays neighbourhood-level data across 5 tabs: Overview, Housing, Safety,
|
||||||
|
Demographics, and Amenities. Each tab provides interactive choropleth maps,
|
||||||
|
KPI cards, and supporting charts.
|
||||||
|
"""
|
||||||
|
|
||||||
import dash
|
import dash
|
||||||
import dash_mantine_components as dmc
|
import dash_mantine_components as dmc
|
||||||
from dash import dcc, html
|
from dash import dcc
|
||||||
from dash_iconify import DashIconify
|
from dash_iconify import DashIconify
|
||||||
|
|
||||||
from portfolio_app.components import (
|
from portfolio_app.pages.toronto.tabs import (
|
||||||
create_map_controls,
|
create_amenities_tab,
|
||||||
create_metric_cards_row,
|
create_demographics_tab,
|
||||||
create_time_slider,
|
create_housing_tab,
|
||||||
create_year_selector,
|
create_overview_tab,
|
||||||
|
create_safety_tab,
|
||||||
)
|
)
|
||||||
|
|
||||||
dash.register_page(__name__, path="/toronto", name="Toronto Housing")
|
dash.register_page(__name__, path="/toronto", name="Toronto Neighbourhoods")
|
||||||
|
|
||||||
# Metric options for the purchase market
|
# Tab configuration
|
||||||
PURCHASE_METRIC_OPTIONS = [
|
TAB_CONFIG = [
|
||||||
{"label": "Average Price", "value": "avg_price"},
|
|
||||||
{"label": "Median Price", "value": "median_price"},
|
|
||||||
{"label": "Sales Volume", "value": "sales_count"},
|
|
||||||
{"label": "Days on Market", "value": "avg_dom"},
|
|
||||||
]
|
|
||||||
|
|
||||||
# Metric options for the rental market
|
|
||||||
RENTAL_METRIC_OPTIONS = [
|
|
||||||
{"label": "Average Rent", "value": "avg_rent"},
|
|
||||||
{"label": "Vacancy Rate", "value": "vacancy_rate"},
|
|
||||||
{"label": "Rental Universe", "value": "rental_universe"},
|
|
||||||
]
|
|
||||||
|
|
||||||
# Sample metrics for KPI cards (will be populated by callbacks)
|
|
||||||
SAMPLE_METRICS = [
|
|
||||||
{
|
{
|
||||||
"title": "Avg. Price",
|
"value": "overview",
|
||||||
"value": 1125000,
|
"label": "Overview",
|
||||||
"delta": 2.3,
|
"icon": "tabler:chart-pie",
|
||||||
"prefix": "$",
|
"color": "blue",
|
||||||
"format_spec": ",.0f",
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"title": "Sales Volume",
|
"value": "housing",
|
||||||
"value": 4850,
|
"label": "Housing",
|
||||||
"delta": -5.1,
|
"icon": "tabler:home",
|
||||||
"format_spec": ",",
|
"color": "teal",
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"title": "Avg. DOM",
|
"value": "safety",
|
||||||
"value": 18,
|
"label": "Safety",
|
||||||
"delta": 3,
|
"icon": "tabler:shield-check",
|
||||||
"suffix": " days",
|
"color": "orange",
|
||||||
"positive_is_good": False,
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"title": "Avg. Rent",
|
"value": "demographics",
|
||||||
"value": 2450,
|
"label": "Demographics",
|
||||||
"delta": 4.2,
|
"icon": "tabler:users",
|
||||||
"prefix": "$",
|
"color": "violet",
|
||||||
"format_spec": ",.0f",
|
},
|
||||||
|
{
|
||||||
|
"value": "amenities",
|
||||||
|
"label": "Amenities",
|
||||||
|
"icon": "tabler:trees",
|
||||||
|
"color": "green",
|
||||||
},
|
},
|
||||||
]
|
]
|
||||||
|
|
||||||
@@ -67,9 +61,9 @@ def create_header() -> dmc.Group:
|
|||||||
[
|
[
|
||||||
dmc.Stack(
|
dmc.Stack(
|
||||||
[
|
[
|
||||||
dmc.Title("Toronto Housing Dashboard", order=1),
|
dmc.Title("Toronto Neighbourhood Dashboard", order=1),
|
||||||
dmc.Text(
|
dmc.Text(
|
||||||
"Real estate market analysis for the Greater Toronto Area",
|
"Explore livability across 158 Toronto neighbourhoods",
|
||||||
c="dimmed",
|
c="dimmed",
|
||||||
),
|
),
|
||||||
],
|
],
|
||||||
@@ -88,11 +82,17 @@ def create_header() -> dmc.Group:
|
|||||||
),
|
),
|
||||||
href="/toronto/methodology",
|
href="/toronto/methodology",
|
||||||
),
|
),
|
||||||
create_year_selector(
|
dmc.Select(
|
||||||
id_prefix="toronto",
|
id="toronto-year-select",
|
||||||
min_year=2020,
|
data=[
|
||||||
default_year=2024,
|
{"value": "2021", "label": "2021"},
|
||||||
label="Year",
|
{"value": "2022", "label": "2022"},
|
||||||
|
{"value": "2023", "label": "2023"},
|
||||||
|
],
|
||||||
|
value="2021",
|
||||||
|
label="Census Year",
|
||||||
|
size="sm",
|
||||||
|
w=120,
|
||||||
),
|
),
|
||||||
],
|
],
|
||||||
gap="md",
|
gap="md",
|
||||||
@@ -103,187 +103,100 @@ def create_header() -> dmc.Group:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def create_kpi_section() -> dmc.Box:
|
def create_neighbourhood_selector() -> dmc.Paper:
|
||||||
"""Create the KPI metrics row."""
|
"""Create the neighbourhood search/select component."""
|
||||||
return dmc.Box(
|
|
||||||
children=[
|
|
||||||
dmc.Title("Key Metrics", order=3, size="h4", mb="sm"),
|
|
||||||
html.Div(
|
|
||||||
id="toronto-kpi-cards",
|
|
||||||
children=[
|
|
||||||
create_metric_cards_row(SAMPLE_METRICS, id_prefix="toronto-kpi")
|
|
||||||
],
|
|
||||||
),
|
|
||||||
],
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def create_purchase_map_section() -> dmc.Grid:
|
|
||||||
"""Create the purchase market choropleth section."""
|
|
||||||
return dmc.Grid(
|
|
||||||
[
|
|
||||||
dmc.GridCol(
|
|
||||||
create_map_controls(
|
|
||||||
id_prefix="purchase-map",
|
|
||||||
metric_options=PURCHASE_METRIC_OPTIONS,
|
|
||||||
default_metric="avg_price",
|
|
||||||
),
|
|
||||||
span={"base": 12, "md": 3},
|
|
||||||
),
|
|
||||||
dmc.GridCol(
|
|
||||||
dmc.Paper(
|
|
||||||
children=[
|
|
||||||
dcc.Graph(
|
|
||||||
id="purchase-choropleth",
|
|
||||||
config={"scrollZoom": True},
|
|
||||||
style={"height": "500px"},
|
|
||||||
),
|
|
||||||
],
|
|
||||||
p="xs",
|
|
||||||
radius="sm",
|
|
||||||
withBorder=True,
|
|
||||||
),
|
|
||||||
span={"base": 12, "md": 9},
|
|
||||||
),
|
|
||||||
],
|
|
||||||
gutter="md",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def create_rental_map_section() -> dmc.Grid:
|
|
||||||
"""Create the rental market choropleth section."""
|
|
||||||
return dmc.Grid(
|
|
||||||
[
|
|
||||||
dmc.GridCol(
|
|
||||||
create_map_controls(
|
|
||||||
id_prefix="rental-map",
|
|
||||||
metric_options=RENTAL_METRIC_OPTIONS,
|
|
||||||
default_metric="avg_rent",
|
|
||||||
),
|
|
||||||
span={"base": 12, "md": 3},
|
|
||||||
),
|
|
||||||
dmc.GridCol(
|
|
||||||
dmc.Paper(
|
|
||||||
children=[
|
|
||||||
dcc.Graph(
|
|
||||||
id="rental-choropleth",
|
|
||||||
config={"scrollZoom": True},
|
|
||||||
style={"height": "500px"},
|
|
||||||
),
|
|
||||||
],
|
|
||||||
p="xs",
|
|
||||||
radius="sm",
|
|
||||||
withBorder=True,
|
|
||||||
),
|
|
||||||
span={"base": 12, "md": 9},
|
|
||||||
),
|
|
||||||
],
|
|
||||||
gutter="md",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def create_time_series_section() -> dmc.Grid:
|
|
||||||
"""Create the time series charts section."""
|
|
||||||
return dmc.Grid(
|
|
||||||
[
|
|
||||||
dmc.GridCol(
|
|
||||||
dmc.Paper(
|
|
||||||
children=[
|
|
||||||
dmc.Title("Price Trends", order=4, size="h5", mb="sm"),
|
|
||||||
dcc.Graph(
|
|
||||||
id="price-time-series",
|
|
||||||
config={"displayModeBar": False},
|
|
||||||
style={"height": "350px"},
|
|
||||||
),
|
|
||||||
],
|
|
||||||
p="md",
|
|
||||||
radius="sm",
|
|
||||||
withBorder=True,
|
|
||||||
),
|
|
||||||
span={"base": 12, "md": 6},
|
|
||||||
),
|
|
||||||
dmc.GridCol(
|
|
||||||
dmc.Paper(
|
|
||||||
children=[
|
|
||||||
dmc.Title("Sales Volume", order=4, size="h5", mb="sm"),
|
|
||||||
dcc.Graph(
|
|
||||||
id="volume-time-series",
|
|
||||||
config={"displayModeBar": False},
|
|
||||||
style={"height": "350px"},
|
|
||||||
),
|
|
||||||
],
|
|
||||||
p="md",
|
|
||||||
radius="sm",
|
|
||||||
withBorder=True,
|
|
||||||
),
|
|
||||||
span={"base": 12, "md": 6},
|
|
||||||
),
|
|
||||||
],
|
|
||||||
gutter="md",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def create_market_comparison_section() -> dmc.Paper:
|
|
||||||
"""Create the market comparison chart section."""
|
|
||||||
return dmc.Paper(
|
return dmc.Paper(
|
||||||
children=[
|
|
||||||
dmc.Group(
|
dmc.Group(
|
||||||
[
|
[
|
||||||
dmc.Title("Market Indicators", order=4, size="h5"),
|
DashIconify(icon="tabler:search", width=20, color="gray"),
|
||||||
create_time_slider(
|
dmc.Select(
|
||||||
id_prefix="market-comparison",
|
id="toronto-neighbourhood-select",
|
||||||
min_year=2020,
|
placeholder="Search neighbourhoods...",
|
||||||
label="",
|
searchable=True,
|
||||||
|
clearable=True,
|
||||||
|
data=[], # Populated by callback
|
||||||
|
style={"flex": 1},
|
||||||
|
),
|
||||||
|
dmc.Button(
|
||||||
|
"Compare",
|
||||||
|
id="toronto-compare-btn",
|
||||||
|
leftSection=DashIconify(icon="tabler:git-compare", width=16),
|
||||||
|
variant="light",
|
||||||
|
disabled=True,
|
||||||
),
|
),
|
||||||
],
|
],
|
||||||
justify="space-between",
|
gap="sm",
|
||||||
align="center",
|
|
||||||
mb="md",
|
|
||||||
),
|
),
|
||||||
dcc.Graph(
|
p="sm",
|
||||||
id="market-comparison-chart",
|
|
||||||
config={"displayModeBar": False},
|
|
||||||
style={"height": "400px"},
|
|
||||||
),
|
|
||||||
],
|
|
||||||
p="md",
|
|
||||||
radius="sm",
|
radius="sm",
|
||||||
withBorder=True,
|
withBorder=True,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def create_tab_navigation() -> dmc.Tabs:
|
||||||
|
"""Create the tab navigation with icons."""
|
||||||
|
return dmc.Tabs(
|
||||||
|
[
|
||||||
|
dmc.TabsList(
|
||||||
|
[
|
||||||
|
dmc.TabsTab(
|
||||||
|
dmc.Group(
|
||||||
|
[
|
||||||
|
DashIconify(icon=tab["icon"], width=18),
|
||||||
|
dmc.Text(tab["label"], size="sm"),
|
||||||
|
],
|
||||||
|
gap="xs",
|
||||||
|
),
|
||||||
|
value=tab["value"],
|
||||||
|
)
|
||||||
|
for tab in TAB_CONFIG
|
||||||
|
],
|
||||||
|
grow=True,
|
||||||
|
),
|
||||||
|
# Tab panels
|
||||||
|
dmc.TabsPanel(create_overview_tab(), value="overview", pt="md"),
|
||||||
|
dmc.TabsPanel(create_housing_tab(), value="housing", pt="md"),
|
||||||
|
dmc.TabsPanel(create_safety_tab(), value="safety", pt="md"),
|
||||||
|
dmc.TabsPanel(create_demographics_tab(), value="demographics", pt="md"),
|
||||||
|
dmc.TabsPanel(create_amenities_tab(), value="amenities", pt="md"),
|
||||||
|
],
|
||||||
|
id="toronto-tabs",
|
||||||
|
value="overview",
|
||||||
|
variant="default",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def create_data_notice() -> dmc.Alert:
|
def create_data_notice() -> dmc.Alert:
|
||||||
"""Create a notice about data availability."""
|
"""Create a notice about data sources."""
|
||||||
return dmc.Alert(
|
return dmc.Alert(
|
||||||
children=[
|
children=[
|
||||||
dmc.Text(
|
dmc.Text(
|
||||||
"This dashboard displays Toronto neighbourhood and CMHC rental data. "
|
"Data from Toronto Open Data (Census 2021, Crime Statistics) and "
|
||||||
"Sample data is shown for demonstration purposes.",
|
"CMHC Rental Market Reports. Click neighbourhoods on the map for details.",
|
||||||
size="sm",
|
size="sm",
|
||||||
),
|
),
|
||||||
],
|
],
|
||||||
title="Data Notice",
|
title="Data Sources",
|
||||||
color="blue",
|
color="blue",
|
||||||
variant="light",
|
variant="light",
|
||||||
|
icon=DashIconify(icon="tabler:info-circle", width=20),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# Store for selected neighbourhood
|
||||||
|
neighbourhood_store = dcc.Store(id="toronto-selected-neighbourhood", data=None)
|
||||||
|
|
||||||
# Register callbacks
|
# Register callbacks
|
||||||
from portfolio_app.pages.toronto import callbacks # noqa: E402, F401
|
from portfolio_app.pages.toronto import callbacks # noqa: E402, F401
|
||||||
|
|
||||||
layout = dmc.Container(
|
layout = dmc.Container(
|
||||||
dmc.Stack(
|
dmc.Stack(
|
||||||
[
|
[
|
||||||
|
neighbourhood_store,
|
||||||
create_header(),
|
create_header(),
|
||||||
create_data_notice(),
|
create_data_notice(),
|
||||||
create_kpi_section(),
|
create_neighbourhood_selector(),
|
||||||
dmc.Divider(my="md", label="Purchase Market", labelPosition="center"),
|
create_tab_navigation(),
|
||||||
create_purchase_map_section(),
|
|
||||||
dmc.Divider(my="md", label="Rental Market", labelPosition="center"),
|
|
||||||
create_rental_map_section(),
|
|
||||||
dmc.Divider(my="md", label="Trends", labelPosition="center"),
|
|
||||||
create_time_series_section(),
|
|
||||||
create_market_comparison_section(),
|
|
||||||
dmc.Space(h=40),
|
dmc.Space(h=40),
|
||||||
],
|
],
|
||||||
gap="lg",
|
gap="lg",
|
||||||
|
|||||||
15
portfolio_app/pages/toronto/tabs/__init__.py
Normal file
15
portfolio_app/pages/toronto/tabs/__init__.py
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
"""Tab modules for Toronto Neighbourhood Dashboard."""
|
||||||
|
|
||||||
|
from .amenities import create_amenities_tab
|
||||||
|
from .demographics import create_demographics_tab
|
||||||
|
from .housing import create_housing_tab
|
||||||
|
from .overview import create_overview_tab
|
||||||
|
from .safety import create_safety_tab
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"create_overview_tab",
|
||||||
|
"create_housing_tab",
|
||||||
|
"create_safety_tab",
|
||||||
|
"create_demographics_tab",
|
||||||
|
"create_amenities_tab",
|
||||||
|
]
|
||||||
207
portfolio_app/pages/toronto/tabs/amenities.py
Normal file
207
portfolio_app/pages/toronto/tabs/amenities.py
Normal file
@@ -0,0 +1,207 @@
|
|||||||
|
"""Amenities tab for Toronto Neighbourhood Dashboard.
|
||||||
|
|
||||||
|
Displays parks, schools, transit, and other amenity metrics.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import dash_mantine_components as dmc
|
||||||
|
from dash import dcc
|
||||||
|
|
||||||
|
|
||||||
|
def create_amenities_tab() -> dmc.Stack:
|
||||||
|
"""Create the Amenities tab layout.
|
||||||
|
|
||||||
|
Layout:
|
||||||
|
- Choropleth map (amenity score) | KPI cards
|
||||||
|
- Amenity breakdown chart | Amenity comparison radar
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Tab content as a Mantine Stack component.
|
||||||
|
"""
|
||||||
|
return dmc.Stack(
|
||||||
|
[
|
||||||
|
# Main content: Map + KPIs
|
||||||
|
dmc.Grid(
|
||||||
|
[
|
||||||
|
# Choropleth map
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Group(
|
||||||
|
[
|
||||||
|
dmc.Title(
|
||||||
|
"Neighbourhood Amenities",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
),
|
||||||
|
dmc.Select(
|
||||||
|
id="amenities-metric-select",
|
||||||
|
data=[
|
||||||
|
{
|
||||||
|
"value": "amenity_score",
|
||||||
|
"label": "Amenity Score",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"value": "parks_per_capita",
|
||||||
|
"label": "Parks per 1K",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"value": "schools_per_capita",
|
||||||
|
"label": "Schools per 1K",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"value": "transit_score",
|
||||||
|
"label": "Transit Score",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
value="amenity_score",
|
||||||
|
size="sm",
|
||||||
|
w=180,
|
||||||
|
),
|
||||||
|
],
|
||||||
|
justify="space-between",
|
||||||
|
mb="sm",
|
||||||
|
),
|
||||||
|
dcc.Graph(
|
||||||
|
id="amenities-choropleth",
|
||||||
|
config={
|
||||||
|
"scrollZoom": True,
|
||||||
|
"displayModeBar": False,
|
||||||
|
},
|
||||||
|
style={"height": "450px"},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
span={"base": 12, "lg": 8},
|
||||||
|
),
|
||||||
|
# KPI cards
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Stack(
|
||||||
|
[
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Text(
|
||||||
|
"City Amenity Score", size="xs", c="dimmed"
|
||||||
|
),
|
||||||
|
dmc.Title(
|
||||||
|
id="amenities-city-score",
|
||||||
|
children="68",
|
||||||
|
order=2,
|
||||||
|
),
|
||||||
|
dmc.Text(
|
||||||
|
"Out of 100",
|
||||||
|
size="sm",
|
||||||
|
c="dimmed",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Text("Total Parks", size="xs", c="dimmed"),
|
||||||
|
dmc.Title(
|
||||||
|
id="amenities-total-parks",
|
||||||
|
children="1,500+",
|
||||||
|
order=2,
|
||||||
|
),
|
||||||
|
dmc.Text(
|
||||||
|
id="amenities-park-area",
|
||||||
|
children="8,000+ hectares",
|
||||||
|
size="sm",
|
||||||
|
c="green",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Text(
|
||||||
|
"Selected Neighbourhood",
|
||||||
|
size="xs",
|
||||||
|
c="dimmed",
|
||||||
|
),
|
||||||
|
dmc.Title(
|
||||||
|
id="amenities-selected-name",
|
||||||
|
children="Click map to select",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
),
|
||||||
|
dmc.Stack(
|
||||||
|
id="amenities-selected-details",
|
||||||
|
children=[
|
||||||
|
dmc.Text("—", c="dimmed"),
|
||||||
|
],
|
||||||
|
gap="xs",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gap="md",
|
||||||
|
),
|
||||||
|
span={"base": 12, "lg": 4},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gutter="md",
|
||||||
|
),
|
||||||
|
# Supporting charts
|
||||||
|
dmc.Grid(
|
||||||
|
[
|
||||||
|
# Amenity breakdown
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Title(
|
||||||
|
"Amenity Breakdown",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
mb="sm",
|
||||||
|
),
|
||||||
|
dcc.Graph(
|
||||||
|
id="amenities-breakdown-chart",
|
||||||
|
config={"displayModeBar": False},
|
||||||
|
style={"height": "300px"},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
span={"base": 12, "md": 6},
|
||||||
|
),
|
||||||
|
# Amenity comparison radar
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Title(
|
||||||
|
"Amenity Comparison",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
mb="sm",
|
||||||
|
),
|
||||||
|
dcc.Graph(
|
||||||
|
id="amenities-radar-chart",
|
||||||
|
config={"displayModeBar": False},
|
||||||
|
style={"height": "300px"},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
span={"base": 12, "md": 6},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gutter="md",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gap="md",
|
||||||
|
)
|
||||||
211
portfolio_app/pages/toronto/tabs/demographics.py
Normal file
211
portfolio_app/pages/toronto/tabs/demographics.py
Normal file
@@ -0,0 +1,211 @@
|
|||||||
|
"""Demographics tab for Toronto Neighbourhood Dashboard.
|
||||||
|
|
||||||
|
Displays population, income, age, and diversity metrics.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import dash_mantine_components as dmc
|
||||||
|
from dash import dcc
|
||||||
|
|
||||||
|
|
||||||
|
def create_demographics_tab() -> dmc.Stack:
|
||||||
|
"""Create the Demographics tab layout.
|
||||||
|
|
||||||
|
Layout:
|
||||||
|
- Choropleth map (demographic metric) | KPI cards
|
||||||
|
- Age distribution chart | Income distribution chart
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Tab content as a Mantine Stack component.
|
||||||
|
"""
|
||||||
|
return dmc.Stack(
|
||||||
|
[
|
||||||
|
# Main content: Map + KPIs
|
||||||
|
dmc.Grid(
|
||||||
|
[
|
||||||
|
# Choropleth map
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Group(
|
||||||
|
[
|
||||||
|
dmc.Title(
|
||||||
|
"Neighbourhood Demographics",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
),
|
||||||
|
dmc.Select(
|
||||||
|
id="demographics-metric-select",
|
||||||
|
data=[
|
||||||
|
{
|
||||||
|
"value": "population",
|
||||||
|
"label": "Population",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"value": "median_income",
|
||||||
|
"label": "Median Income",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"value": "median_age",
|
||||||
|
"label": "Median Age",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"value": "diversity_index",
|
||||||
|
"label": "Diversity Index",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
value="population",
|
||||||
|
size="sm",
|
||||||
|
w=180,
|
||||||
|
),
|
||||||
|
],
|
||||||
|
justify="space-between",
|
||||||
|
mb="sm",
|
||||||
|
),
|
||||||
|
dcc.Graph(
|
||||||
|
id="demographics-choropleth",
|
||||||
|
config={
|
||||||
|
"scrollZoom": True,
|
||||||
|
"displayModeBar": False,
|
||||||
|
},
|
||||||
|
style={"height": "450px"},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
span={"base": 12, "lg": 8},
|
||||||
|
),
|
||||||
|
# KPI cards
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Stack(
|
||||||
|
[
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Text(
|
||||||
|
"City Population", size="xs", c="dimmed"
|
||||||
|
),
|
||||||
|
dmc.Title(
|
||||||
|
id="demographics-city-pop",
|
||||||
|
children="2.79M",
|
||||||
|
order=2,
|
||||||
|
),
|
||||||
|
dmc.Text(
|
||||||
|
id="demographics-pop-change",
|
||||||
|
children="+2.3% since 2016",
|
||||||
|
size="sm",
|
||||||
|
c="green",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Text(
|
||||||
|
"Median Household Income",
|
||||||
|
size="xs",
|
||||||
|
c="dimmed",
|
||||||
|
),
|
||||||
|
dmc.Title(
|
||||||
|
id="demographics-city-income",
|
||||||
|
children="$84,000",
|
||||||
|
order=2,
|
||||||
|
),
|
||||||
|
dmc.Text(
|
||||||
|
"City average",
|
||||||
|
size="sm",
|
||||||
|
c="dimmed",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Text(
|
||||||
|
"Selected Neighbourhood",
|
||||||
|
size="xs",
|
||||||
|
c="dimmed",
|
||||||
|
),
|
||||||
|
dmc.Title(
|
||||||
|
id="demographics-selected-name",
|
||||||
|
children="Click map to select",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
),
|
||||||
|
dmc.Stack(
|
||||||
|
id="demographics-selected-details",
|
||||||
|
children=[
|
||||||
|
dmc.Text("—", c="dimmed"),
|
||||||
|
],
|
||||||
|
gap="xs",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gap="md",
|
||||||
|
),
|
||||||
|
span={"base": 12, "lg": 4},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gutter="md",
|
||||||
|
),
|
||||||
|
# Supporting charts
|
||||||
|
dmc.Grid(
|
||||||
|
[
|
||||||
|
# Age distribution
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Title(
|
||||||
|
"Age Distribution",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
mb="sm",
|
||||||
|
),
|
||||||
|
dcc.Graph(
|
||||||
|
id="demographics-age-chart",
|
||||||
|
config={"displayModeBar": False},
|
||||||
|
style={"height": "300px"},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
span={"base": 12, "md": 6},
|
||||||
|
),
|
||||||
|
# Income distribution
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Title(
|
||||||
|
"Income Distribution",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
mb="sm",
|
||||||
|
),
|
||||||
|
dcc.Graph(
|
||||||
|
id="demographics-income-chart",
|
||||||
|
config={"displayModeBar": False},
|
||||||
|
style={"height": "300px"},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
span={"base": 12, "md": 6},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gutter="md",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gap="md",
|
||||||
|
)
|
||||||
209
portfolio_app/pages/toronto/tabs/housing.py
Normal file
209
portfolio_app/pages/toronto/tabs/housing.py
Normal file
@@ -0,0 +1,209 @@
|
|||||||
|
"""Housing tab for Toronto Neighbourhood Dashboard.
|
||||||
|
|
||||||
|
Displays affordability metrics, rent trends, and housing indicators.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import dash_mantine_components as dmc
|
||||||
|
from dash import dcc
|
||||||
|
|
||||||
|
|
||||||
|
def create_housing_tab() -> dmc.Stack:
|
||||||
|
"""Create the Housing tab layout.
|
||||||
|
|
||||||
|
Layout:
|
||||||
|
- Choropleth map (affordability index) | KPI cards
|
||||||
|
- Rent trend line chart | Dwelling types breakdown
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Tab content as a Mantine Stack component.
|
||||||
|
"""
|
||||||
|
return dmc.Stack(
|
||||||
|
[
|
||||||
|
# Main content: Map + KPIs
|
||||||
|
dmc.Grid(
|
||||||
|
[
|
||||||
|
# Choropleth map
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Group(
|
||||||
|
[
|
||||||
|
dmc.Title(
|
||||||
|
"Housing Affordability",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
),
|
||||||
|
dmc.Select(
|
||||||
|
id="housing-metric-select",
|
||||||
|
data=[
|
||||||
|
{
|
||||||
|
"value": "affordability_index",
|
||||||
|
"label": "Affordability Index",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"value": "avg_rent_2bed",
|
||||||
|
"label": "Avg Rent (2BR)",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"value": "rent_to_income_pct",
|
||||||
|
"label": "Rent-to-Income %",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"value": "vacancy_rate",
|
||||||
|
"label": "Vacancy Rate",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
value="affordability_index",
|
||||||
|
size="sm",
|
||||||
|
w=180,
|
||||||
|
),
|
||||||
|
],
|
||||||
|
justify="space-between",
|
||||||
|
mb="sm",
|
||||||
|
),
|
||||||
|
dcc.Graph(
|
||||||
|
id="housing-choropleth",
|
||||||
|
config={
|
||||||
|
"scrollZoom": True,
|
||||||
|
"displayModeBar": False,
|
||||||
|
},
|
||||||
|
style={"height": "450px"},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
span={"base": 12, "lg": 8},
|
||||||
|
),
|
||||||
|
# KPI cards
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Stack(
|
||||||
|
[
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Text(
|
||||||
|
"City Avg 2BR Rent", size="xs", c="dimmed"
|
||||||
|
),
|
||||||
|
dmc.Title(
|
||||||
|
id="housing-city-rent",
|
||||||
|
children="$2,450",
|
||||||
|
order=2,
|
||||||
|
),
|
||||||
|
dmc.Text(
|
||||||
|
id="housing-rent-change",
|
||||||
|
children="+4.2% YoY",
|
||||||
|
size="sm",
|
||||||
|
c="red",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Text(
|
||||||
|
"City Avg Vacancy", size="xs", c="dimmed"
|
||||||
|
),
|
||||||
|
dmc.Title(
|
||||||
|
id="housing-city-vacancy",
|
||||||
|
children="1.8%",
|
||||||
|
order=2,
|
||||||
|
),
|
||||||
|
dmc.Text(
|
||||||
|
"Below healthy rate (3%)",
|
||||||
|
size="sm",
|
||||||
|
c="orange",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Text(
|
||||||
|
"Selected Neighbourhood",
|
||||||
|
size="xs",
|
||||||
|
c="dimmed",
|
||||||
|
),
|
||||||
|
dmc.Title(
|
||||||
|
id="housing-selected-name",
|
||||||
|
children="Click map to select",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
),
|
||||||
|
dmc.Stack(
|
||||||
|
id="housing-selected-details",
|
||||||
|
children=[
|
||||||
|
dmc.Text("—", c="dimmed"),
|
||||||
|
],
|
||||||
|
gap="xs",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gap="md",
|
||||||
|
),
|
||||||
|
span={"base": 12, "lg": 4},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gutter="md",
|
||||||
|
),
|
||||||
|
# Supporting charts
|
||||||
|
dmc.Grid(
|
||||||
|
[
|
||||||
|
# Rent trend
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Title(
|
||||||
|
"Rent Trends (5 Year)",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
mb="sm",
|
||||||
|
),
|
||||||
|
dcc.Graph(
|
||||||
|
id="housing-trend-chart",
|
||||||
|
config={"displayModeBar": False},
|
||||||
|
style={"height": "300px"},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
span={"base": 12, "md": 6},
|
||||||
|
),
|
||||||
|
# Dwelling types
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Title(
|
||||||
|
"Dwelling Types",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
mb="sm",
|
||||||
|
),
|
||||||
|
dcc.Graph(
|
||||||
|
id="housing-types-chart",
|
||||||
|
config={"displayModeBar": False},
|
||||||
|
style={"height": "300px"},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
span={"base": 12, "md": 6},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gutter="md",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gap="md",
|
||||||
|
)
|
||||||
233
portfolio_app/pages/toronto/tabs/overview.py
Normal file
233
portfolio_app/pages/toronto/tabs/overview.py
Normal file
@@ -0,0 +1,233 @@
|
|||||||
|
"""Overview tab for Toronto Neighbourhood Dashboard.
|
||||||
|
|
||||||
|
Displays composite livability score with safety, affordability, and amenity components.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import dash_mantine_components as dmc
|
||||||
|
from dash import dcc, html
|
||||||
|
|
||||||
|
|
||||||
|
def create_overview_tab() -> dmc.Stack:
|
||||||
|
"""Create the Overview tab layout.
|
||||||
|
|
||||||
|
Layout:
|
||||||
|
- Choropleth map (livability score) | KPI cards
|
||||||
|
- Top/Bottom 10 bar chart | Income vs Crime scatter
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Tab content as a Mantine Stack component.
|
||||||
|
"""
|
||||||
|
return dmc.Stack(
|
||||||
|
[
|
||||||
|
# Main content: Map + KPIs
|
||||||
|
dmc.Grid(
|
||||||
|
[
|
||||||
|
# Choropleth map
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Group(
|
||||||
|
[
|
||||||
|
dmc.Title(
|
||||||
|
"Neighbourhood Livability",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
),
|
||||||
|
dmc.Select(
|
||||||
|
id="overview-metric-select",
|
||||||
|
data=[
|
||||||
|
{
|
||||||
|
"value": "livability_score",
|
||||||
|
"label": "Livability Score",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"value": "safety_score",
|
||||||
|
"label": "Safety Score",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"value": "affordability_score",
|
||||||
|
"label": "Affordability Score",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"value": "amenity_score",
|
||||||
|
"label": "Amenity Score",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
value="livability_score",
|
||||||
|
size="sm",
|
||||||
|
w=180,
|
||||||
|
),
|
||||||
|
],
|
||||||
|
justify="space-between",
|
||||||
|
mb="sm",
|
||||||
|
),
|
||||||
|
dcc.Graph(
|
||||||
|
id="overview-choropleth",
|
||||||
|
config={
|
||||||
|
"scrollZoom": True,
|
||||||
|
"displayModeBar": False,
|
||||||
|
},
|
||||||
|
style={"height": "450px"},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
span={"base": 12, "lg": 8},
|
||||||
|
),
|
||||||
|
# KPI cards
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Stack(
|
||||||
|
[
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Text("City Average", size="xs", c="dimmed"),
|
||||||
|
dmc.Title(
|
||||||
|
id="overview-city-avg",
|
||||||
|
children="72",
|
||||||
|
order=2,
|
||||||
|
),
|
||||||
|
dmc.Text("Livability Score", size="sm", fw=500),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Text(
|
||||||
|
"Selected Neighbourhood",
|
||||||
|
size="xs",
|
||||||
|
c="dimmed",
|
||||||
|
),
|
||||||
|
dmc.Title(
|
||||||
|
id="overview-selected-name",
|
||||||
|
children="Click map to select",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
),
|
||||||
|
html.Div(
|
||||||
|
id="overview-selected-scores",
|
||||||
|
children=[
|
||||||
|
dmc.Text("—", c="dimmed"),
|
||||||
|
],
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Text(
|
||||||
|
"Score Components", size="xs", c="dimmed"
|
||||||
|
),
|
||||||
|
dmc.Stack(
|
||||||
|
[
|
||||||
|
dmc.Group(
|
||||||
|
[
|
||||||
|
dmc.Text("Safety", size="sm"),
|
||||||
|
dmc.Text(
|
||||||
|
"30%",
|
||||||
|
size="sm",
|
||||||
|
c="dimmed",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
justify="space-between",
|
||||||
|
),
|
||||||
|
dmc.Group(
|
||||||
|
[
|
||||||
|
dmc.Text(
|
||||||
|
"Affordability", size="sm"
|
||||||
|
),
|
||||||
|
dmc.Text(
|
||||||
|
"40%",
|
||||||
|
size="sm",
|
||||||
|
c="dimmed",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
justify="space-between",
|
||||||
|
),
|
||||||
|
dmc.Group(
|
||||||
|
[
|
||||||
|
dmc.Text(
|
||||||
|
"Amenities", size="sm"
|
||||||
|
),
|
||||||
|
dmc.Text(
|
||||||
|
"30%",
|
||||||
|
size="sm",
|
||||||
|
c="dimmed",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
justify="space-between",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gap="xs",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gap="md",
|
||||||
|
),
|
||||||
|
span={"base": 12, "lg": 4},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gutter="md",
|
||||||
|
),
|
||||||
|
# Supporting charts
|
||||||
|
dmc.Grid(
|
||||||
|
[
|
||||||
|
# Top/Bottom rankings
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Title(
|
||||||
|
"Top & Bottom Neighbourhoods",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
mb="sm",
|
||||||
|
),
|
||||||
|
dcc.Graph(
|
||||||
|
id="overview-rankings-chart",
|
||||||
|
config={"displayModeBar": False},
|
||||||
|
style={"height": "300px"},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
span={"base": 12, "md": 6},
|
||||||
|
),
|
||||||
|
# Scatter plot
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Title(
|
||||||
|
"Income vs Safety",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
mb="sm",
|
||||||
|
),
|
||||||
|
dcc.Graph(
|
||||||
|
id="overview-scatter-chart",
|
||||||
|
config={"displayModeBar": False},
|
||||||
|
style={"height": "300px"},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
span={"base": 12, "md": 6},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gutter="md",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gap="md",
|
||||||
|
)
|
||||||
211
portfolio_app/pages/toronto/tabs/safety.py
Normal file
211
portfolio_app/pages/toronto/tabs/safety.py
Normal file
@@ -0,0 +1,211 @@
|
|||||||
|
"""Safety tab for Toronto Neighbourhood Dashboard.
|
||||||
|
|
||||||
|
Displays crime statistics, trends, and safety indicators.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import dash_mantine_components as dmc
|
||||||
|
from dash import dcc
|
||||||
|
|
||||||
|
|
||||||
|
def create_safety_tab() -> dmc.Stack:
|
||||||
|
"""Create the Safety tab layout.
|
||||||
|
|
||||||
|
Layout:
|
||||||
|
- Choropleth map (crime rate) | KPI cards
|
||||||
|
- Crime trend line chart | Crime by type breakdown
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Tab content as a Mantine Stack component.
|
||||||
|
"""
|
||||||
|
return dmc.Stack(
|
||||||
|
[
|
||||||
|
# Main content: Map + KPIs
|
||||||
|
dmc.Grid(
|
||||||
|
[
|
||||||
|
# Choropleth map
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Group(
|
||||||
|
[
|
||||||
|
dmc.Title(
|
||||||
|
"Crime Rate by Neighbourhood",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
),
|
||||||
|
dmc.Select(
|
||||||
|
id="safety-metric-select",
|
||||||
|
data=[
|
||||||
|
{
|
||||||
|
"value": "total_crime_rate",
|
||||||
|
"label": "Total Crime Rate",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"value": "violent_crime_rate",
|
||||||
|
"label": "Violent Crime",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"value": "property_crime_rate",
|
||||||
|
"label": "Property Crime",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"value": "theft_rate",
|
||||||
|
"label": "Theft",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
value="total_crime_rate",
|
||||||
|
size="sm",
|
||||||
|
w=180,
|
||||||
|
),
|
||||||
|
],
|
||||||
|
justify="space-between",
|
||||||
|
mb="sm",
|
||||||
|
),
|
||||||
|
dcc.Graph(
|
||||||
|
id="safety-choropleth",
|
||||||
|
config={
|
||||||
|
"scrollZoom": True,
|
||||||
|
"displayModeBar": False,
|
||||||
|
},
|
||||||
|
style={"height": "450px"},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
span={"base": 12, "lg": 8},
|
||||||
|
),
|
||||||
|
# KPI cards
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Stack(
|
||||||
|
[
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Text(
|
||||||
|
"City Crime Rate", size="xs", c="dimmed"
|
||||||
|
),
|
||||||
|
dmc.Title(
|
||||||
|
id="safety-city-rate",
|
||||||
|
children="4,250",
|
||||||
|
order=2,
|
||||||
|
),
|
||||||
|
dmc.Text(
|
||||||
|
id="safety-rate-change",
|
||||||
|
children="-2.1% YoY",
|
||||||
|
size="sm",
|
||||||
|
c="green",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Text(
|
||||||
|
"Total Incidents (2023)",
|
||||||
|
size="xs",
|
||||||
|
c="dimmed",
|
||||||
|
),
|
||||||
|
dmc.Title(
|
||||||
|
id="safety-total-incidents",
|
||||||
|
children="125,430",
|
||||||
|
order=2,
|
||||||
|
),
|
||||||
|
dmc.Text(
|
||||||
|
"Per 100,000 residents",
|
||||||
|
size="sm",
|
||||||
|
c="dimmed",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Text(
|
||||||
|
"Selected Neighbourhood",
|
||||||
|
size="xs",
|
||||||
|
c="dimmed",
|
||||||
|
),
|
||||||
|
dmc.Title(
|
||||||
|
id="safety-selected-name",
|
||||||
|
children="Click map to select",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
),
|
||||||
|
dmc.Stack(
|
||||||
|
id="safety-selected-details",
|
||||||
|
children=[
|
||||||
|
dmc.Text("—", c="dimmed"),
|
||||||
|
],
|
||||||
|
gap="xs",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gap="md",
|
||||||
|
),
|
||||||
|
span={"base": 12, "lg": 4},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gutter="md",
|
||||||
|
),
|
||||||
|
# Supporting charts
|
||||||
|
dmc.Grid(
|
||||||
|
[
|
||||||
|
# Crime trend
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Title(
|
||||||
|
"Crime Trends (5 Year)",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
mb="sm",
|
||||||
|
),
|
||||||
|
dcc.Graph(
|
||||||
|
id="safety-trend-chart",
|
||||||
|
config={"displayModeBar": False},
|
||||||
|
style={"height": "300px"},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
span={"base": 12, "md": 6},
|
||||||
|
),
|
||||||
|
# Crime by type
|
||||||
|
dmc.GridCol(
|
||||||
|
dmc.Paper(
|
||||||
|
[
|
||||||
|
dmc.Title(
|
||||||
|
"Crime by Category",
|
||||||
|
order=4,
|
||||||
|
size="h5",
|
||||||
|
mb="sm",
|
||||||
|
),
|
||||||
|
dcc.Graph(
|
||||||
|
id="safety-types-chart",
|
||||||
|
config={"displayModeBar": False},
|
||||||
|
style={"height": "300px"},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
p="md",
|
||||||
|
radius="sm",
|
||||||
|
withBorder=True,
|
||||||
|
),
|
||||||
|
span={"base": 12, "md": 6},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gutter="md",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
gap="md",
|
||||||
|
)
|
||||||
@@ -57,6 +57,7 @@ class TorontoOpenDataParser:
|
|||||||
self._cache_dir = cache_dir
|
self._cache_dir = cache_dir
|
||||||
self._timeout = timeout
|
self._timeout = timeout
|
||||||
self._client: httpx.Client | None = None
|
self._client: httpx.Client | None = None
|
||||||
|
self._neighbourhood_name_map: dict[str, int] | None = None
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def client(self) -> httpx.Client:
|
def client(self) -> httpx.Client:
|
||||||
@@ -75,6 +76,63 @@ class TorontoOpenDataParser:
|
|||||||
self._client.close()
|
self._client.close()
|
||||||
self._client = None
|
self._client = None
|
||||||
|
|
||||||
|
def _get_neighbourhood_name_map(self) -> dict[str, int]:
|
||||||
|
"""Build and cache a mapping of neighbourhood names to IDs.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Dictionary mapping normalized neighbourhood names to area_id.
|
||||||
|
"""
|
||||||
|
if self._neighbourhood_name_map is not None:
|
||||||
|
return self._neighbourhood_name_map
|
||||||
|
|
||||||
|
neighbourhoods = self.get_neighbourhoods()
|
||||||
|
self._neighbourhood_name_map = {}
|
||||||
|
|
||||||
|
for n in neighbourhoods:
|
||||||
|
# Add multiple variations of the name for flexible matching
|
||||||
|
name_lower = n.area_name.lower().strip()
|
||||||
|
self._neighbourhood_name_map[name_lower] = n.area_id
|
||||||
|
|
||||||
|
# Also add without common suffixes/prefixes
|
||||||
|
for suffix in [" neighbourhood", " area", "-"]:
|
||||||
|
if suffix in name_lower:
|
||||||
|
alt_name = name_lower.replace(suffix, "").strip()
|
||||||
|
self._neighbourhood_name_map[alt_name] = n.area_id
|
||||||
|
|
||||||
|
logger.debug(
|
||||||
|
f"Built neighbourhood name map with {len(self._neighbourhood_name_map)} entries"
|
||||||
|
)
|
||||||
|
return self._neighbourhood_name_map
|
||||||
|
|
||||||
|
def _match_neighbourhood_id(self, name: str) -> int | None:
|
||||||
|
"""Match a neighbourhood name to its ID.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
name: Neighbourhood name from census data.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Neighbourhood ID or None if not found.
|
||||||
|
"""
|
||||||
|
name_map = self._get_neighbourhood_name_map()
|
||||||
|
name_lower = name.lower().strip()
|
||||||
|
|
||||||
|
# Direct match
|
||||||
|
if name_lower in name_map:
|
||||||
|
return name_map[name_lower]
|
||||||
|
|
||||||
|
# Try removing parenthetical content
|
||||||
|
if "(" in name_lower:
|
||||||
|
base_name = name_lower.split("(")[0].strip()
|
||||||
|
if base_name in name_map:
|
||||||
|
return name_map[base_name]
|
||||||
|
|
||||||
|
# Try fuzzy matching with first few chars
|
||||||
|
for key, area_id in name_map.items():
|
||||||
|
if key.startswith(name_lower[:10]) or name_lower.startswith(key[:10]):
|
||||||
|
return area_id
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
def __enter__(self) -> "TorontoOpenDataParser":
|
def __enter__(self) -> "TorontoOpenDataParser":
|
||||||
return self
|
return self
|
||||||
|
|
||||||
@@ -254,11 +312,30 @@ class TorontoOpenDataParser:
|
|||||||
logger.info(f"Parsed {len(records)} neighbourhoods")
|
logger.info(f"Parsed {len(records)} neighbourhoods")
|
||||||
return records
|
return records
|
||||||
|
|
||||||
|
# Mapping of indicator names to CensusRecord fields
|
||||||
|
# Keys are partial matches (case-insensitive) found in the "Characteristic" column
|
||||||
|
CENSUS_INDICATOR_MAPPING: dict[str, str] = {
|
||||||
|
"population, 2021": "population",
|
||||||
|
"population, 2016": "population",
|
||||||
|
"population density per square kilometre": "population_density",
|
||||||
|
"median total income of household": "median_household_income",
|
||||||
|
"average total income of household": "average_household_income",
|
||||||
|
"unemployment rate": "unemployment_rate",
|
||||||
|
"bachelor's degree or higher": "pct_bachelors_or_higher",
|
||||||
|
"owner": "pct_owner_occupied",
|
||||||
|
"renter": "pct_renter_occupied",
|
||||||
|
"median age": "median_age",
|
||||||
|
"average value of dwellings": "average_dwelling_value",
|
||||||
|
}
|
||||||
|
|
||||||
def get_census_profiles(self, year: int = 2021) -> list[CensusRecord]:
|
def get_census_profiles(self, year: int = 2021) -> list[CensusRecord]:
|
||||||
"""Fetch neighbourhood census profiles.
|
"""Fetch neighbourhood census profiles.
|
||||||
|
|
||||||
Note: Census profile data structure varies by year. This method
|
The Toronto Open Data neighbourhood profiles dataset is pivoted:
|
||||||
extracts key demographic indicators where available.
|
- Rows are demographic indicators (e.g., "Population", "Median Income")
|
||||||
|
- Columns are neighbourhoods (e.g., "Agincourt North", "Alderwood")
|
||||||
|
|
||||||
|
This method transposes the data to create one CensusRecord per neighbourhood.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
year: Census year (2016 or 2021).
|
year: Census year (2016 or 2021).
|
||||||
@@ -266,7 +343,6 @@ class TorontoOpenDataParser:
|
|||||||
Returns:
|
Returns:
|
||||||
List of validated CensusRecord objects.
|
List of validated CensusRecord objects.
|
||||||
"""
|
"""
|
||||||
# Census profiles are typically in CSV/datastore format
|
|
||||||
try:
|
try:
|
||||||
raw_records = self._fetch_csv_as_json(
|
raw_records = self._fetch_csv_as_json(
|
||||||
self.DATASETS["neighbourhood_profiles"]
|
self.DATASETS["neighbourhood_profiles"]
|
||||||
@@ -275,14 +351,120 @@ class TorontoOpenDataParser:
|
|||||||
logger.warning(f"Could not fetch census profiles: {e}")
|
logger.warning(f"Could not fetch census profiles: {e}")
|
||||||
return []
|
return []
|
||||||
|
|
||||||
# Census profiles are pivoted - rows are indicators, columns are neighbourhoods
|
if not raw_records:
|
||||||
# This requires special handling based on the actual data structure
|
logger.warning("Census profiles dataset is empty")
|
||||||
|
return []
|
||||||
|
|
||||||
logger.info(f"Fetched {len(raw_records)} census profile rows")
|
logger.info(f"Fetched {len(raw_records)} census profile rows")
|
||||||
|
|
||||||
# For now, return empty list - actual implementation depends on data structure
|
# Find the characteristic/indicator column name
|
||||||
# TODO: Implement census profile parsing based on actual data format
|
sample_row = raw_records[0]
|
||||||
|
char_col = None
|
||||||
|
for col in sample_row:
|
||||||
|
col_lower = col.lower()
|
||||||
|
if "characteristic" in col_lower or "category" in col_lower:
|
||||||
|
char_col = col
|
||||||
|
break
|
||||||
|
|
||||||
|
if not char_col:
|
||||||
|
# Try common column names
|
||||||
|
for candidate in ["Characteristic", "Category", "Topic", "_id"]:
|
||||||
|
if candidate in sample_row:
|
||||||
|
char_col = candidate
|
||||||
|
break
|
||||||
|
|
||||||
|
if not char_col:
|
||||||
|
logger.warning("Could not find characteristic column in census data")
|
||||||
return []
|
return []
|
||||||
|
|
||||||
|
# Identify neighbourhood columns (exclude metadata columns)
|
||||||
|
exclude_cols = {
|
||||||
|
char_col,
|
||||||
|
"_id",
|
||||||
|
"Topic",
|
||||||
|
"Data Source",
|
||||||
|
"Characteristic",
|
||||||
|
"Category",
|
||||||
|
}
|
||||||
|
neighbourhood_cols = [col for col in sample_row if col not in exclude_cols]
|
||||||
|
|
||||||
|
logger.info(f"Found {len(neighbourhood_cols)} neighbourhood columns")
|
||||||
|
|
||||||
|
# Build a lookup: neighbourhood_name -> {field: value}
|
||||||
|
neighbourhood_data: dict[str, dict[str, Decimal | int | None]] = {
|
||||||
|
col: {} for col in neighbourhood_cols
|
||||||
|
}
|
||||||
|
|
||||||
|
# Process each row to extract indicator values
|
||||||
|
for row in raw_records:
|
||||||
|
characteristic = str(row.get(char_col, "")).lower().strip()
|
||||||
|
|
||||||
|
# Check if this row matches any indicator we care about
|
||||||
|
for indicator_pattern, field_name in self.CENSUS_INDICATOR_MAPPING.items():
|
||||||
|
if indicator_pattern in characteristic:
|
||||||
|
# Extract values for each neighbourhood
|
||||||
|
for col in neighbourhood_cols:
|
||||||
|
value = row.get(col)
|
||||||
|
if value is not None and value != "":
|
||||||
|
try:
|
||||||
|
# Clean and convert value
|
||||||
|
str_val = str(value).replace(",", "").replace("$", "")
|
||||||
|
str_val = str_val.replace("%", "").strip()
|
||||||
|
if str_val and str_val not in ("x", "X", "F", ".."):
|
||||||
|
numeric_val = Decimal(str_val)
|
||||||
|
# Only store if not already set (first match wins)
|
||||||
|
if field_name not in neighbourhood_data[col]:
|
||||||
|
neighbourhood_data[col][
|
||||||
|
field_name
|
||||||
|
] = numeric_val
|
||||||
|
except (ValueError, TypeError):
|
||||||
|
pass
|
||||||
|
break # Move to next row after matching
|
||||||
|
|
||||||
|
# Convert to CensusRecord objects
|
||||||
|
records = []
|
||||||
|
unmatched = []
|
||||||
|
|
||||||
|
for neighbourhood_name, data in neighbourhood_data.items():
|
||||||
|
if not data:
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Match neighbourhood name to ID
|
||||||
|
neighbourhood_id = self._match_neighbourhood_id(neighbourhood_name)
|
||||||
|
if neighbourhood_id is None:
|
||||||
|
unmatched.append(neighbourhood_name)
|
||||||
|
continue
|
||||||
|
|
||||||
|
try:
|
||||||
|
pop_val = data.get("population")
|
||||||
|
population = int(pop_val) if pop_val is not None else None
|
||||||
|
|
||||||
|
record = CensusRecord(
|
||||||
|
neighbourhood_id=neighbourhood_id,
|
||||||
|
census_year=year,
|
||||||
|
population=population,
|
||||||
|
population_density=data.get("population_density"),
|
||||||
|
median_household_income=data.get("median_household_income"),
|
||||||
|
average_household_income=data.get("average_household_income"),
|
||||||
|
unemployment_rate=data.get("unemployment_rate"),
|
||||||
|
pct_bachelors_or_higher=data.get("pct_bachelors_or_higher"),
|
||||||
|
pct_owner_occupied=data.get("pct_owner_occupied"),
|
||||||
|
pct_renter_occupied=data.get("pct_renter_occupied"),
|
||||||
|
median_age=data.get("median_age"),
|
||||||
|
average_dwelling_value=data.get("average_dwelling_value"),
|
||||||
|
)
|
||||||
|
records.append(record)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug(f"Skipping neighbourhood {neighbourhood_name}: {e}")
|
||||||
|
|
||||||
|
if unmatched:
|
||||||
|
logger.warning(
|
||||||
|
f"Could not match {len(unmatched)} neighbourhoods: {unmatched[:5]}..."
|
||||||
|
)
|
||||||
|
|
||||||
|
logger.info(f"Parsed {len(records)} census records for year {year}")
|
||||||
|
return records
|
||||||
|
|
||||||
def get_parks(self) -> list[AmenityRecord]:
|
def get_parks(self) -> list[AmenityRecord]:
|
||||||
"""Fetch park locations.
|
"""Fetch park locations.
|
||||||
|
|
||||||
|
|||||||
33
portfolio_app/toronto/services/__init__.py
Normal file
33
portfolio_app/toronto/services/__init__.py
Normal file
@@ -0,0 +1,33 @@
|
|||||||
|
"""Data service layer for Toronto neighbourhood dashboard."""
|
||||||
|
|
||||||
|
from .geometry_service import (
|
||||||
|
get_cmhc_zones_geojson,
|
||||||
|
get_neighbourhoods_geojson,
|
||||||
|
)
|
||||||
|
from .neighbourhood_service import (
|
||||||
|
get_amenities_data,
|
||||||
|
get_city_averages,
|
||||||
|
get_demographics_data,
|
||||||
|
get_housing_data,
|
||||||
|
get_neighbourhood_details,
|
||||||
|
get_neighbourhood_list,
|
||||||
|
get_overview_data,
|
||||||
|
get_rankings,
|
||||||
|
get_safety_data,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
# Neighbourhood data
|
||||||
|
"get_overview_data",
|
||||||
|
"get_housing_data",
|
||||||
|
"get_safety_data",
|
||||||
|
"get_demographics_data",
|
||||||
|
"get_amenities_data",
|
||||||
|
"get_neighbourhood_details",
|
||||||
|
"get_neighbourhood_list",
|
||||||
|
"get_rankings",
|
||||||
|
"get_city_averages",
|
||||||
|
# Geometry
|
||||||
|
"get_neighbourhoods_geojson",
|
||||||
|
"get_cmhc_zones_geojson",
|
||||||
|
]
|
||||||
176
portfolio_app/toronto/services/geometry_service.py
Normal file
176
portfolio_app/toronto/services/geometry_service.py
Normal file
@@ -0,0 +1,176 @@
|
|||||||
|
"""Service layer for generating GeoJSON from PostGIS geometry."""
|
||||||
|
|
||||||
|
import json
|
||||||
|
from functools import lru_cache
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pandas as pd
|
||||||
|
from sqlalchemy import text
|
||||||
|
|
||||||
|
from portfolio_app.toronto.models import get_engine
|
||||||
|
|
||||||
|
|
||||||
|
def _execute_query(sql: str, params: dict[str, Any] | None = None) -> pd.DataFrame:
|
||||||
|
"""Execute SQL query and return DataFrame."""
|
||||||
|
engine = get_engine()
|
||||||
|
with engine.connect() as conn:
|
||||||
|
return pd.read_sql(text(sql), conn, params=params)
|
||||||
|
|
||||||
|
|
||||||
|
@lru_cache(maxsize=8)
|
||||||
|
def get_neighbourhoods_geojson(year: int = 2021) -> dict[str, Any]:
|
||||||
|
"""Get GeoJSON FeatureCollection for all neighbourhoods.
|
||||||
|
|
||||||
|
Queries mart_neighbourhood_overview for geometries and basic properties.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
year: Year to query for joining properties.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
GeoJSON FeatureCollection dictionary.
|
||||||
|
"""
|
||||||
|
# Query geometries with ST_AsGeoJSON
|
||||||
|
sql = """
|
||||||
|
SELECT
|
||||||
|
neighbourhood_id,
|
||||||
|
neighbourhood_name,
|
||||||
|
ST_AsGeoJSON(geometry)::json as geom,
|
||||||
|
population,
|
||||||
|
livability_score
|
||||||
|
FROM mart_neighbourhood_overview
|
||||||
|
WHERE year = :year
|
||||||
|
AND geometry IS NOT NULL
|
||||||
|
"""
|
||||||
|
|
||||||
|
try:
|
||||||
|
df = _execute_query(sql, {"year": year})
|
||||||
|
except Exception:
|
||||||
|
# Table might not exist or have data yet
|
||||||
|
return _empty_geojson()
|
||||||
|
|
||||||
|
if df.empty:
|
||||||
|
return _empty_geojson()
|
||||||
|
|
||||||
|
# Build GeoJSON features
|
||||||
|
features = []
|
||||||
|
for _, row in df.iterrows():
|
||||||
|
geom = row["geom"]
|
||||||
|
if geom is None:
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Handle geometry that might be a string or dict
|
||||||
|
if isinstance(geom, str):
|
||||||
|
geom = json.loads(geom)
|
||||||
|
|
||||||
|
feature = {
|
||||||
|
"type": "Feature",
|
||||||
|
"id": row["neighbourhood_id"],
|
||||||
|
"properties": {
|
||||||
|
"neighbourhood_id": int(row["neighbourhood_id"]),
|
||||||
|
"neighbourhood_name": row["neighbourhood_name"],
|
||||||
|
"population": int(row["population"])
|
||||||
|
if pd.notna(row["population"])
|
||||||
|
else None,
|
||||||
|
"livability_score": float(row["livability_score"])
|
||||||
|
if pd.notna(row["livability_score"])
|
||||||
|
else None,
|
||||||
|
},
|
||||||
|
"geometry": geom,
|
||||||
|
}
|
||||||
|
features.append(feature)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"type": "FeatureCollection",
|
||||||
|
"features": features,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@lru_cache(maxsize=4)
|
||||||
|
def get_cmhc_zones_geojson() -> dict[str, Any]:
|
||||||
|
"""Get GeoJSON FeatureCollection for CMHC zones.
|
||||||
|
|
||||||
|
Queries dim_cmhc_zone for zone geometries.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
GeoJSON FeatureCollection dictionary.
|
||||||
|
"""
|
||||||
|
sql = """
|
||||||
|
SELECT
|
||||||
|
zone_code,
|
||||||
|
zone_name,
|
||||||
|
ST_AsGeoJSON(geometry)::json as geom
|
||||||
|
FROM dim_cmhc_zone
|
||||||
|
WHERE geometry IS NOT NULL
|
||||||
|
"""
|
||||||
|
|
||||||
|
try:
|
||||||
|
df = _execute_query(sql, {})
|
||||||
|
except Exception:
|
||||||
|
return _empty_geojson()
|
||||||
|
|
||||||
|
if df.empty:
|
||||||
|
return _empty_geojson()
|
||||||
|
|
||||||
|
features = []
|
||||||
|
for _, row in df.iterrows():
|
||||||
|
geom = row["geom"]
|
||||||
|
if geom is None:
|
||||||
|
continue
|
||||||
|
|
||||||
|
if isinstance(geom, str):
|
||||||
|
geom = json.loads(geom)
|
||||||
|
|
||||||
|
feature = {
|
||||||
|
"type": "Feature",
|
||||||
|
"id": row["zone_code"],
|
||||||
|
"properties": {
|
||||||
|
"zone_code": row["zone_code"],
|
||||||
|
"zone_name": row["zone_name"],
|
||||||
|
},
|
||||||
|
"geometry": geom,
|
||||||
|
}
|
||||||
|
features.append(feature)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"type": "FeatureCollection",
|
||||||
|
"features": features,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def get_neighbourhood_geometry(neighbourhood_id: int) -> dict[str, Any] | None:
|
||||||
|
"""Get GeoJSON geometry for a single neighbourhood.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
neighbourhood_id: The neighbourhood ID.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
GeoJSON geometry dict, or None if not found.
|
||||||
|
"""
|
||||||
|
sql = """
|
||||||
|
SELECT ST_AsGeoJSON(geometry)::json as geom
|
||||||
|
FROM dim_neighbourhood
|
||||||
|
WHERE neighbourhood_id = :neighbourhood_id
|
||||||
|
AND geometry IS NOT NULL
|
||||||
|
"""
|
||||||
|
|
||||||
|
try:
|
||||||
|
df = _execute_query(sql, {"neighbourhood_id": neighbourhood_id})
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
|
||||||
|
if df.empty:
|
||||||
|
return None
|
||||||
|
|
||||||
|
geom = df.iloc[0]["geom"]
|
||||||
|
if isinstance(geom, str):
|
||||||
|
result: dict[str, Any] = json.loads(geom)
|
||||||
|
return result
|
||||||
|
return dict(geom) if geom is not None else None
|
||||||
|
|
||||||
|
|
||||||
|
def _empty_geojson() -> dict[str, Any]:
|
||||||
|
"""Return an empty GeoJSON FeatureCollection."""
|
||||||
|
return {
|
||||||
|
"type": "FeatureCollection",
|
||||||
|
"features": [],
|
||||||
|
}
|
||||||
392
portfolio_app/toronto/services/neighbourhood_service.py
Normal file
392
portfolio_app/toronto/services/neighbourhood_service.py
Normal file
@@ -0,0 +1,392 @@
|
|||||||
|
"""Service layer for querying neighbourhood data from dbt marts."""
|
||||||
|
|
||||||
|
from functools import lru_cache
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pandas as pd
|
||||||
|
from sqlalchemy import text
|
||||||
|
|
||||||
|
from portfolio_app.toronto.models import get_engine
|
||||||
|
|
||||||
|
|
||||||
|
def _execute_query(sql: str, params: dict[str, Any] | None = None) -> pd.DataFrame:
|
||||||
|
"""Execute SQL query and return DataFrame.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
sql: SQL query string.
|
||||||
|
params: Query parameters.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
pandas DataFrame with results, or empty DataFrame on error.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
engine = get_engine()
|
||||||
|
with engine.connect() as conn:
|
||||||
|
return pd.read_sql(text(sql), conn, params=params)
|
||||||
|
except Exception:
|
||||||
|
# Return empty DataFrame on connection or query error
|
||||||
|
return pd.DataFrame()
|
||||||
|
|
||||||
|
|
||||||
|
def get_overview_data(year: int = 2021) -> pd.DataFrame:
|
||||||
|
"""Get overview data for all neighbourhoods.
|
||||||
|
|
||||||
|
Queries mart_neighbourhood_overview for livability scores and components.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
year: Census year to query.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
DataFrame with columns: neighbourhood_id, neighbourhood_name,
|
||||||
|
livability_score, safety_score, affordability_score, amenity_score,
|
||||||
|
population, median_household_income, etc.
|
||||||
|
"""
|
||||||
|
sql = """
|
||||||
|
SELECT
|
||||||
|
neighbourhood_id,
|
||||||
|
neighbourhood_name,
|
||||||
|
year,
|
||||||
|
population,
|
||||||
|
median_household_income,
|
||||||
|
livability_score,
|
||||||
|
safety_score,
|
||||||
|
affordability_score,
|
||||||
|
amenity_score,
|
||||||
|
crime_rate_per_100k,
|
||||||
|
rent_to_income_pct,
|
||||||
|
avg_rent_2bed,
|
||||||
|
total_amenities_per_1000
|
||||||
|
FROM mart_neighbourhood_overview
|
||||||
|
WHERE year = :year
|
||||||
|
ORDER BY livability_score DESC NULLS LAST
|
||||||
|
"""
|
||||||
|
return _execute_query(sql, {"year": year})
|
||||||
|
|
||||||
|
|
||||||
|
def get_housing_data(year: int = 2021) -> pd.DataFrame:
|
||||||
|
"""Get housing data for all neighbourhoods.
|
||||||
|
|
||||||
|
Queries mart_neighbourhood_housing for affordability metrics.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
year: Year to query.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
DataFrame with columns: neighbourhood_id, neighbourhood_name,
|
||||||
|
avg_rent_2bed, vacancy_rate, rent_to_income_pct, affordability_index, etc.
|
||||||
|
"""
|
||||||
|
sql = """
|
||||||
|
SELECT
|
||||||
|
neighbourhood_id,
|
||||||
|
neighbourhood_name,
|
||||||
|
year,
|
||||||
|
pct_owner_occupied,
|
||||||
|
pct_renter_occupied,
|
||||||
|
average_dwelling_value,
|
||||||
|
median_household_income,
|
||||||
|
avg_rent_bachelor,
|
||||||
|
avg_rent_1bed,
|
||||||
|
avg_rent_2bed,
|
||||||
|
avg_rent_3bed,
|
||||||
|
vacancy_rate,
|
||||||
|
total_rental_units,
|
||||||
|
rent_to_income_pct,
|
||||||
|
is_affordable,
|
||||||
|
affordability_index,
|
||||||
|
rent_yoy_change_pct,
|
||||||
|
income_quintile
|
||||||
|
FROM mart_neighbourhood_housing
|
||||||
|
WHERE year = :year
|
||||||
|
ORDER BY affordability_index ASC NULLS LAST
|
||||||
|
"""
|
||||||
|
return _execute_query(sql, {"year": year})
|
||||||
|
|
||||||
|
|
||||||
|
def get_safety_data(year: int = 2021) -> pd.DataFrame:
|
||||||
|
"""Get safety/crime data for all neighbourhoods.
|
||||||
|
|
||||||
|
Queries mart_neighbourhood_safety for crime statistics.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
year: Year to query.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
DataFrame with columns: neighbourhood_id, neighbourhood_name,
|
||||||
|
total_crime_rate, violent_crime_rate, property_crime_rate, etc.
|
||||||
|
"""
|
||||||
|
sql = """
|
||||||
|
SELECT
|
||||||
|
neighbourhood_id,
|
||||||
|
neighbourhood_name,
|
||||||
|
year,
|
||||||
|
total_crimes,
|
||||||
|
crime_rate_per_100k as total_crime_rate,
|
||||||
|
violent_crimes,
|
||||||
|
violent_crime_rate,
|
||||||
|
property_crimes,
|
||||||
|
property_crime_rate,
|
||||||
|
theft_crimes,
|
||||||
|
theft_rate,
|
||||||
|
crime_yoy_change_pct,
|
||||||
|
crime_trend
|
||||||
|
FROM mart_neighbourhood_safety
|
||||||
|
WHERE year = :year
|
||||||
|
ORDER BY total_crime_rate ASC NULLS LAST
|
||||||
|
"""
|
||||||
|
return _execute_query(sql, {"year": year})
|
||||||
|
|
||||||
|
|
||||||
|
def get_demographics_data(year: int = 2021) -> pd.DataFrame:
|
||||||
|
"""Get demographic data for all neighbourhoods.
|
||||||
|
|
||||||
|
Queries mart_neighbourhood_demographics for population/income metrics.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
year: Census year to query.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
DataFrame with columns: neighbourhood_id, neighbourhood_name,
|
||||||
|
population, median_age, median_income, diversity_index, etc.
|
||||||
|
"""
|
||||||
|
sql = """
|
||||||
|
SELECT
|
||||||
|
neighbourhood_id,
|
||||||
|
neighbourhood_name,
|
||||||
|
census_year as year,
|
||||||
|
population,
|
||||||
|
population_density,
|
||||||
|
population_change_pct,
|
||||||
|
median_household_income,
|
||||||
|
average_household_income,
|
||||||
|
income_quintile,
|
||||||
|
median_age,
|
||||||
|
pct_under_18,
|
||||||
|
pct_18_to_64,
|
||||||
|
pct_65_plus,
|
||||||
|
pct_bachelors_or_higher,
|
||||||
|
unemployment_rate,
|
||||||
|
diversity_index
|
||||||
|
FROM mart_neighbourhood_demographics
|
||||||
|
WHERE census_year = :year
|
||||||
|
ORDER BY population DESC NULLS LAST
|
||||||
|
"""
|
||||||
|
return _execute_query(sql, {"year": year})
|
||||||
|
|
||||||
|
|
||||||
|
def get_amenities_data(year: int = 2021) -> pd.DataFrame:
|
||||||
|
"""Get amenities data for all neighbourhoods.
|
||||||
|
|
||||||
|
Queries mart_neighbourhood_amenities for parks, schools, transit.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
year: Year to query.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
DataFrame with columns: neighbourhood_id, neighbourhood_name,
|
||||||
|
amenity_score, parks_per_capita, schools_per_capita, transit_score, etc.
|
||||||
|
"""
|
||||||
|
sql = """
|
||||||
|
SELECT
|
||||||
|
neighbourhood_id,
|
||||||
|
neighbourhood_name,
|
||||||
|
year,
|
||||||
|
park_count,
|
||||||
|
parks_per_1000,
|
||||||
|
school_count,
|
||||||
|
schools_per_1000,
|
||||||
|
childcare_count,
|
||||||
|
childcare_per_1000,
|
||||||
|
total_amenities,
|
||||||
|
total_amenities_per_1000,
|
||||||
|
amenity_score,
|
||||||
|
amenity_rank
|
||||||
|
FROM mart_neighbourhood_amenities
|
||||||
|
WHERE year = :year
|
||||||
|
ORDER BY amenity_score DESC NULLS LAST
|
||||||
|
"""
|
||||||
|
return _execute_query(sql, {"year": year})
|
||||||
|
|
||||||
|
|
||||||
|
def get_neighbourhood_details(
|
||||||
|
neighbourhood_id: int, year: int = 2021
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Get detailed data for a single neighbourhood.
|
||||||
|
|
||||||
|
Combines data from all mart tables for a complete neighbourhood profile.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
neighbourhood_id: The neighbourhood ID.
|
||||||
|
year: Year to query.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Dictionary with all metrics for the neighbourhood.
|
||||||
|
"""
|
||||||
|
sql = """
|
||||||
|
SELECT
|
||||||
|
o.neighbourhood_id,
|
||||||
|
o.neighbourhood_name,
|
||||||
|
o.year,
|
||||||
|
o.population,
|
||||||
|
o.median_household_income,
|
||||||
|
o.livability_score,
|
||||||
|
o.safety_score,
|
||||||
|
o.affordability_score,
|
||||||
|
o.amenity_score,
|
||||||
|
s.total_crimes,
|
||||||
|
s.crime_rate_per_100k,
|
||||||
|
s.violent_crime_rate,
|
||||||
|
s.property_crime_rate,
|
||||||
|
h.avg_rent_2bed,
|
||||||
|
h.vacancy_rate,
|
||||||
|
h.rent_to_income_pct,
|
||||||
|
h.affordability_index,
|
||||||
|
h.pct_owner_occupied,
|
||||||
|
h.pct_renter_occupied,
|
||||||
|
d.median_age,
|
||||||
|
d.diversity_index,
|
||||||
|
d.unemployment_rate,
|
||||||
|
d.pct_bachelors_or_higher,
|
||||||
|
a.park_count,
|
||||||
|
a.school_count,
|
||||||
|
a.total_amenities
|
||||||
|
FROM mart_neighbourhood_overview o
|
||||||
|
LEFT JOIN mart_neighbourhood_safety s
|
||||||
|
ON o.neighbourhood_id = s.neighbourhood_id
|
||||||
|
AND o.year = s.year
|
||||||
|
LEFT JOIN mart_neighbourhood_housing h
|
||||||
|
ON o.neighbourhood_id = h.neighbourhood_id
|
||||||
|
AND o.year = h.year
|
||||||
|
LEFT JOIN mart_neighbourhood_demographics d
|
||||||
|
ON o.neighbourhood_id = d.neighbourhood_id
|
||||||
|
AND o.year = d.census_year
|
||||||
|
LEFT JOIN mart_neighbourhood_amenities a
|
||||||
|
ON o.neighbourhood_id = a.neighbourhood_id
|
||||||
|
AND o.year = a.year
|
||||||
|
WHERE o.neighbourhood_id = :neighbourhood_id
|
||||||
|
AND o.year = :year
|
||||||
|
"""
|
||||||
|
df = _execute_query(sql, {"neighbourhood_id": neighbourhood_id, "year": year})
|
||||||
|
|
||||||
|
if df.empty:
|
||||||
|
return {}
|
||||||
|
|
||||||
|
return {str(k): v for k, v in df.iloc[0].to_dict().items()}
|
||||||
|
|
||||||
|
|
||||||
|
@lru_cache(maxsize=32)
|
||||||
|
def get_neighbourhood_list(year: int = 2021) -> list[dict[str, Any]]:
|
||||||
|
"""Get list of all neighbourhoods for dropdown selectors.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
year: Year to query.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of dicts with neighbourhood_id, name, and population.
|
||||||
|
"""
|
||||||
|
sql = """
|
||||||
|
SELECT DISTINCT
|
||||||
|
neighbourhood_id,
|
||||||
|
neighbourhood_name,
|
||||||
|
population
|
||||||
|
FROM mart_neighbourhood_overview
|
||||||
|
WHERE year = :year
|
||||||
|
ORDER BY neighbourhood_name
|
||||||
|
"""
|
||||||
|
df = _execute_query(sql, {"year": year})
|
||||||
|
if df.empty:
|
||||||
|
return []
|
||||||
|
return list(df.to_dict("records")) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
|
||||||
|
def get_rankings(
|
||||||
|
metric: str,
|
||||||
|
year: int = 2021,
|
||||||
|
top_n: int = 10,
|
||||||
|
ascending: bool = True,
|
||||||
|
) -> pd.DataFrame:
|
||||||
|
"""Get top/bottom neighbourhoods for a specific metric.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
metric: Column name to rank by.
|
||||||
|
year: Year to query.
|
||||||
|
top_n: Number of top and bottom records.
|
||||||
|
ascending: If True, rank from lowest to highest (good for crime, rent).
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
DataFrame with top and bottom neighbourhoods.
|
||||||
|
"""
|
||||||
|
# Map metrics to their source tables
|
||||||
|
table_map = {
|
||||||
|
"livability_score": "mart_neighbourhood_overview",
|
||||||
|
"safety_score": "mart_neighbourhood_overview",
|
||||||
|
"affordability_score": "mart_neighbourhood_overview",
|
||||||
|
"amenity_score": "mart_neighbourhood_overview",
|
||||||
|
"crime_rate_per_100k": "mart_neighbourhood_safety",
|
||||||
|
"total_crime_rate": "mart_neighbourhood_safety",
|
||||||
|
"avg_rent_2bed": "mart_neighbourhood_housing",
|
||||||
|
"affordability_index": "mart_neighbourhood_housing",
|
||||||
|
"population": "mart_neighbourhood_demographics",
|
||||||
|
"median_household_income": "mart_neighbourhood_demographics",
|
||||||
|
}
|
||||||
|
|
||||||
|
table = table_map.get(metric, "mart_neighbourhood_overview")
|
||||||
|
year_col = "census_year" if "demographics" in table else "year"
|
||||||
|
|
||||||
|
order = "ASC" if ascending else "DESC"
|
||||||
|
reverse_order = "DESC" if ascending else "ASC"
|
||||||
|
|
||||||
|
sql = f"""
|
||||||
|
(
|
||||||
|
SELECT neighbourhood_id, neighbourhood_name, {metric}, 'bottom' as rank_group
|
||||||
|
FROM {table}
|
||||||
|
WHERE {year_col} = :year AND {metric} IS NOT NULL
|
||||||
|
ORDER BY {metric} {order}
|
||||||
|
LIMIT :top_n
|
||||||
|
)
|
||||||
|
UNION ALL
|
||||||
|
(
|
||||||
|
SELECT neighbourhood_id, neighbourhood_name, {metric}, 'top' as rank_group
|
||||||
|
FROM {table}
|
||||||
|
WHERE {year_col} = :year AND {metric} IS NOT NULL
|
||||||
|
ORDER BY {metric} {reverse_order}
|
||||||
|
LIMIT :top_n
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
return _execute_query(sql, {"year": year, "top_n": top_n})
|
||||||
|
|
||||||
|
|
||||||
|
def get_city_averages(year: int = 2021) -> dict[str, Any]:
|
||||||
|
"""Get city-wide average metrics.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
year: Year to query.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Dictionary with city averages for key metrics.
|
||||||
|
"""
|
||||||
|
sql = """
|
||||||
|
SELECT
|
||||||
|
AVG(livability_score) as avg_livability_score,
|
||||||
|
AVG(safety_score) as avg_safety_score,
|
||||||
|
AVG(affordability_score) as avg_affordability_score,
|
||||||
|
AVG(amenity_score) as avg_amenity_score,
|
||||||
|
SUM(population) as total_population,
|
||||||
|
AVG(median_household_income) as avg_median_income,
|
||||||
|
AVG(crime_rate_per_100k) as avg_crime_rate,
|
||||||
|
AVG(avg_rent_2bed) as avg_rent_2bed,
|
||||||
|
AVG(rent_to_income_pct) as avg_rent_to_income
|
||||||
|
FROM mart_neighbourhood_overview
|
||||||
|
WHERE year = :year
|
||||||
|
"""
|
||||||
|
df = _execute_query(sql, {"year": year})
|
||||||
|
|
||||||
|
if df.empty:
|
||||||
|
return {}
|
||||||
|
|
||||||
|
result: dict[str, Any] = {str(k): v for k, v in df.iloc[0].to_dict().items()}
|
||||||
|
# Round numeric values
|
||||||
|
for key, value in result.items():
|
||||||
|
if pd.notna(value) and isinstance(value, float):
|
||||||
|
result[key] = round(value, 2)
|
||||||
|
|
||||||
|
return result
|
||||||
1
scripts/data/__init__.py
Normal file
1
scripts/data/__init__.py
Normal file
@@ -0,0 +1 @@
|
|||||||
|
"""Data loading scripts for the portfolio app."""
|
||||||
367
scripts/data/load_toronto_data.py
Normal file
367
scripts/data/load_toronto_data.py
Normal file
@@ -0,0 +1,367 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Load Toronto neighbourhood data into the database.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
python scripts/data/load_toronto_data.py [OPTIONS]
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--skip-fetch Skip API fetching, only run dbt
|
||||||
|
--skip-dbt Skip dbt run, only load data
|
||||||
|
--dry-run Show what would be done without executing
|
||||||
|
-v, --verbose Enable verbose logging
|
||||||
|
|
||||||
|
This script orchestrates:
|
||||||
|
1. Fetching data from Toronto Open Data and CMHC APIs
|
||||||
|
2. Loading data into PostgreSQL fact tables
|
||||||
|
3. Running dbt to transform staging -> intermediate -> marts
|
||||||
|
|
||||||
|
Exit codes:
|
||||||
|
0 = Success
|
||||||
|
1 = Error
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import logging
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from datetime import date
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
# Add project root to path
|
||||||
|
PROJECT_ROOT = Path(__file__).parent.parent.parent
|
||||||
|
sys.path.insert(0, str(PROJECT_ROOT))
|
||||||
|
|
||||||
|
from portfolio_app.toronto.loaders import ( # noqa: E402
|
||||||
|
get_session,
|
||||||
|
load_amenities,
|
||||||
|
load_census_data,
|
||||||
|
load_crime_data,
|
||||||
|
load_neighbourhoods,
|
||||||
|
load_time_dimension,
|
||||||
|
)
|
||||||
|
from portfolio_app.toronto.parsers import ( # noqa: E402
|
||||||
|
TorontoOpenDataParser,
|
||||||
|
TorontoPoliceParser,
|
||||||
|
)
|
||||||
|
from portfolio_app.toronto.schemas import Neighbourhood # noqa: E402
|
||||||
|
|
||||||
|
# Configure logging
|
||||||
|
logging.basicConfig(
|
||||||
|
level=logging.INFO,
|
||||||
|
format="%(asctime)s - %(levelname)s - %(message)s",
|
||||||
|
datefmt="%H:%M:%S",
|
||||||
|
)
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class DataPipeline:
|
||||||
|
"""Orchestrates data loading from APIs to database to dbt."""
|
||||||
|
|
||||||
|
def __init__(self, dry_run: bool = False, verbose: bool = False):
|
||||||
|
self.dry_run = dry_run
|
||||||
|
self.verbose = verbose
|
||||||
|
self.stats: dict[str, int] = {}
|
||||||
|
|
||||||
|
if verbose:
|
||||||
|
logging.getLogger().setLevel(logging.DEBUG)
|
||||||
|
|
||||||
|
def fetch_and_load(self) -> bool:
|
||||||
|
"""Fetch data from APIs and load into database.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
True if successful, False otherwise.
|
||||||
|
"""
|
||||||
|
logger.info("Starting data fetch and load pipeline...")
|
||||||
|
|
||||||
|
try:
|
||||||
|
with get_session() as session:
|
||||||
|
# 1. Load time dimension first (for date keys)
|
||||||
|
self._load_time_dimension(session)
|
||||||
|
|
||||||
|
# 2. Load neighbourhoods (required for foreign keys)
|
||||||
|
self._load_neighbourhoods(session)
|
||||||
|
|
||||||
|
# 3. Load census data
|
||||||
|
self._load_census(session)
|
||||||
|
|
||||||
|
# 4. Load crime data
|
||||||
|
self._load_crime(session)
|
||||||
|
|
||||||
|
# 5. Load amenities
|
||||||
|
self._load_amenities(session)
|
||||||
|
|
||||||
|
session.commit()
|
||||||
|
logger.info("All data committed to database")
|
||||||
|
|
||||||
|
self._print_stats()
|
||||||
|
return True
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Pipeline failed: {e}")
|
||||||
|
if self.verbose:
|
||||||
|
import traceback
|
||||||
|
|
||||||
|
traceback.print_exc()
|
||||||
|
return False
|
||||||
|
|
||||||
|
def _load_time_dimension(self, session: Any) -> None:
|
||||||
|
"""Load time dimension with date range for dashboard."""
|
||||||
|
logger.info("Loading time dimension...")
|
||||||
|
|
||||||
|
if self.dry_run:
|
||||||
|
logger.info(
|
||||||
|
" [DRY RUN] Would load time dimension 2019-01-01 to 2025-12-01"
|
||||||
|
)
|
||||||
|
return
|
||||||
|
|
||||||
|
count = load_time_dimension(
|
||||||
|
start_date=date(2019, 1, 1),
|
||||||
|
end_date=date(2025, 12, 1),
|
||||||
|
session=session,
|
||||||
|
)
|
||||||
|
self.stats["time_dimension"] = count
|
||||||
|
logger.info(f" Loaded {count} time dimension records")
|
||||||
|
|
||||||
|
def _load_neighbourhoods(self, session: Any) -> None:
|
||||||
|
"""Fetch and load neighbourhood boundaries."""
|
||||||
|
logger.info("Fetching neighbourhoods from Toronto Open Data...")
|
||||||
|
|
||||||
|
if self.dry_run:
|
||||||
|
logger.info(" [DRY RUN] Would fetch and load neighbourhoods")
|
||||||
|
return
|
||||||
|
|
||||||
|
import json
|
||||||
|
|
||||||
|
parser = TorontoOpenDataParser()
|
||||||
|
raw_neighbourhoods = parser.get_neighbourhoods()
|
||||||
|
|
||||||
|
# Convert NeighbourhoodRecord to Neighbourhood schema
|
||||||
|
neighbourhoods = []
|
||||||
|
for n in raw_neighbourhoods:
|
||||||
|
# Convert GeoJSON geometry dict to WKT if present
|
||||||
|
geometry_wkt = None
|
||||||
|
if n.geometry:
|
||||||
|
# Store as GeoJSON string for PostGIS ST_GeomFromGeoJSON
|
||||||
|
geometry_wkt = json.dumps(n.geometry)
|
||||||
|
|
||||||
|
neighbourhood = Neighbourhood(
|
||||||
|
neighbourhood_id=n.area_id,
|
||||||
|
name=n.area_name,
|
||||||
|
geometry_wkt=geometry_wkt,
|
||||||
|
population=None, # Will be filled from census data
|
||||||
|
land_area_sqkm=None,
|
||||||
|
pop_density_per_sqkm=None,
|
||||||
|
census_year=2021,
|
||||||
|
)
|
||||||
|
neighbourhoods.append(neighbourhood)
|
||||||
|
|
||||||
|
count = load_neighbourhoods(neighbourhoods, session)
|
||||||
|
self.stats["neighbourhoods"] = count
|
||||||
|
logger.info(f" Loaded {count} neighbourhoods")
|
||||||
|
|
||||||
|
def _load_census(self, session: Any) -> None:
|
||||||
|
"""Fetch and load census profile data."""
|
||||||
|
logger.info("Fetching census profiles from Toronto Open Data...")
|
||||||
|
|
||||||
|
if self.dry_run:
|
||||||
|
logger.info(" [DRY RUN] Would fetch and load census data")
|
||||||
|
return
|
||||||
|
|
||||||
|
parser = TorontoOpenDataParser()
|
||||||
|
census_records = parser.get_census_profiles(year=2021)
|
||||||
|
|
||||||
|
if not census_records:
|
||||||
|
logger.warning(" No census records fetched")
|
||||||
|
return
|
||||||
|
|
||||||
|
count = load_census_data(census_records, session)
|
||||||
|
self.stats["census"] = count
|
||||||
|
logger.info(f" Loaded {count} census records")
|
||||||
|
|
||||||
|
def _load_crime(self, session: Any) -> None:
|
||||||
|
"""Fetch and load crime statistics."""
|
||||||
|
logger.info("Fetching crime data from Toronto Police Service...")
|
||||||
|
|
||||||
|
if self.dry_run:
|
||||||
|
logger.info(" [DRY RUN] Would fetch and load crime data")
|
||||||
|
return
|
||||||
|
|
||||||
|
parser = TorontoPoliceParser()
|
||||||
|
crime_records = parser.get_crime_rates()
|
||||||
|
|
||||||
|
if not crime_records:
|
||||||
|
logger.warning(" No crime records fetched")
|
||||||
|
return
|
||||||
|
|
||||||
|
count = load_crime_data(crime_records, session)
|
||||||
|
self.stats["crime"] = count
|
||||||
|
logger.info(f" Loaded {count} crime records")
|
||||||
|
|
||||||
|
def _load_amenities(self, session: Any) -> None:
|
||||||
|
"""Fetch and load amenity data (parks, schools, childcare)."""
|
||||||
|
logger.info("Fetching amenities from Toronto Open Data...")
|
||||||
|
|
||||||
|
if self.dry_run:
|
||||||
|
logger.info(" [DRY RUN] Would fetch and load amenity data")
|
||||||
|
return
|
||||||
|
|
||||||
|
parser = TorontoOpenDataParser()
|
||||||
|
total_count = 0
|
||||||
|
|
||||||
|
# Fetch parks
|
||||||
|
try:
|
||||||
|
parks = parser.get_parks()
|
||||||
|
if parks:
|
||||||
|
count = load_amenities(parks, year=2024, session=session)
|
||||||
|
total_count += count
|
||||||
|
logger.info(f" Loaded {count} park amenities")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f" Failed to load parks: {e}")
|
||||||
|
|
||||||
|
# Fetch schools
|
||||||
|
try:
|
||||||
|
schools = parser.get_schools()
|
||||||
|
if schools:
|
||||||
|
count = load_amenities(schools, year=2024, session=session)
|
||||||
|
total_count += count
|
||||||
|
logger.info(f" Loaded {count} school amenities")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f" Failed to load schools: {e}")
|
||||||
|
|
||||||
|
# Fetch childcare centres
|
||||||
|
try:
|
||||||
|
childcare = parser.get_childcare_centres()
|
||||||
|
if childcare:
|
||||||
|
count = load_amenities(childcare, year=2024, session=session)
|
||||||
|
total_count += count
|
||||||
|
logger.info(f" Loaded {count} childcare amenities")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f" Failed to load childcare: {e}")
|
||||||
|
|
||||||
|
self.stats["amenities"] = total_count
|
||||||
|
|
||||||
|
def run_dbt(self) -> bool:
|
||||||
|
"""Run dbt to transform data.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
True if successful, False otherwise.
|
||||||
|
"""
|
||||||
|
logger.info("Running dbt transformations...")
|
||||||
|
|
||||||
|
dbt_project_dir = PROJECT_ROOT / "dbt"
|
||||||
|
|
||||||
|
if not dbt_project_dir.exists():
|
||||||
|
logger.error(f"dbt project directory not found: {dbt_project_dir}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
if self.dry_run:
|
||||||
|
logger.info(" [DRY RUN] Would run: dbt run")
|
||||||
|
logger.info(" [DRY RUN] Would run: dbt test")
|
||||||
|
return True
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Run dbt models
|
||||||
|
logger.info(" Running dbt run...")
|
||||||
|
result = subprocess.run(
|
||||||
|
["dbt", "run"],
|
||||||
|
cwd=dbt_project_dir,
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
if result.returncode != 0:
|
||||||
|
logger.error(f"dbt run failed:\n{result.stderr}")
|
||||||
|
if self.verbose:
|
||||||
|
logger.debug(f"dbt output:\n{result.stdout}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
logger.info(" dbt run completed successfully")
|
||||||
|
|
||||||
|
# Run dbt tests
|
||||||
|
logger.info(" Running dbt test...")
|
||||||
|
result = subprocess.run(
|
||||||
|
["dbt", "test"],
|
||||||
|
cwd=dbt_project_dir,
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
if result.returncode != 0:
|
||||||
|
logger.warning(f"dbt test had failures:\n{result.stderr}")
|
||||||
|
# Don't fail on test failures, just warn
|
||||||
|
else:
|
||||||
|
logger.info(" dbt test completed successfully")
|
||||||
|
|
||||||
|
return True
|
||||||
|
|
||||||
|
except FileNotFoundError:
|
||||||
|
logger.error(
|
||||||
|
"dbt not found in PATH. Install with: pip install dbt-postgres"
|
||||||
|
)
|
||||||
|
return False
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"dbt execution failed: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
def _print_stats(self) -> None:
|
||||||
|
"""Print loading statistics."""
|
||||||
|
if not self.stats:
|
||||||
|
return
|
||||||
|
|
||||||
|
logger.info("Loading statistics:")
|
||||||
|
for key, count in self.stats.items():
|
||||||
|
logger.info(f" {key}: {count} records")
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
"""Main entry point for the data loading script."""
|
||||||
|
parser = argparse.ArgumentParser(
|
||||||
|
description="Load Toronto neighbourhood data into the database",
|
||||||
|
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||||
|
epilog=__doc__,
|
||||||
|
)
|
||||||
|
|
||||||
|
parser.add_argument(
|
||||||
|
"--skip-fetch",
|
||||||
|
action="store_true",
|
||||||
|
help="Skip API fetching, only run dbt",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--skip-dbt",
|
||||||
|
action="store_true",
|
||||||
|
help="Skip dbt run, only load data",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--dry-run",
|
||||||
|
action="store_true",
|
||||||
|
help="Show what would be done without executing",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"-v",
|
||||||
|
"--verbose",
|
||||||
|
action="store_true",
|
||||||
|
help="Enable verbose logging",
|
||||||
|
)
|
||||||
|
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
if args.skip_fetch and args.skip_dbt:
|
||||||
|
logger.error("Cannot skip both fetch and dbt - nothing to do")
|
||||||
|
return 1
|
||||||
|
|
||||||
|
pipeline = DataPipeline(dry_run=args.dry_run, verbose=args.verbose)
|
||||||
|
|
||||||
|
# Execute pipeline stages
|
||||||
|
if not args.skip_fetch and not pipeline.fetch_and_load():
|
||||||
|
return 1
|
||||||
|
|
||||||
|
if not args.skip_dbt and not pipeline.run_dbt():
|
||||||
|
return 1
|
||||||
|
|
||||||
|
logger.info("Pipeline completed successfully!")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
Reference in New Issue
Block a user