refactor: multi-dashboard structural migration
Some checks failed
CI / lint-and-test (pull_request) Has been cancelled
Some checks failed
CI / lint-and-test (pull_request) Has been cancelled
- Rename dbt project from toronto_housing to portfolio - Restructure dbt models into domain subdirectories: - shared/ for cross-domain dimensions (dim_time) - staging/toronto/, intermediate/toronto/, marts/toronto/ - Update SQLAlchemy models for raw_toronto schema - Add explicit cross-schema FK relationships for FactRentals - Namespace figure factories under figures/toronto/ - Namespace notebooks under notebooks/toronto/ - Update Makefile with domain-specific targets and env loading - Update all documentation for multi-dashboard structure This enables adding new dashboard projects (e.g., /football, /energy) without structural conflicts or naming collisions. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
201
notebooks/toronto/overview/livability_choropleth.ipynb
Normal file
201
notebooks/toronto/overview/livability_choropleth.ipynb
Normal file
@@ -0,0 +1,201 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Livability Score Choropleth Map\n",
|
||||
"\n",
|
||||
"Displays neighbourhood livability scores on an interactive map of Toronto's 158 neighbourhoods."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 1. Data Reference\n",
|
||||
"\n",
|
||||
"### Source Tables\n",
|
||||
"\n",
|
||||
"| Table | Grain | Key Columns |\n",
|
||||
"|-------|-------|-------------|\n",
|
||||
"| `mart_neighbourhood_overview` | neighbourhood × year | livability_score, safety_score, affordability_score, amenity_score, geometry |\n",
|
||||
"\n",
|
||||
"### SQL Query"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"import pandas as pd\n",
|
||||
"from dotenv import load_dotenv\n",
|
||||
"from sqlalchemy import create_engine\n",
|
||||
"\n",
|
||||
"# Load .env from project root\n",
|
||||
"load_dotenv(\"../../.env\")\n",
|
||||
"\n",
|
||||
"engine = create_engine(os.environ[\"DATABASE_URL\"])\n",
|
||||
"\n",
|
||||
"query = \"\"\"\n",
|
||||
"SELECT\n",
|
||||
" neighbourhood_id,\n",
|
||||
" neighbourhood_name,\n",
|
||||
" geometry,\n",
|
||||
" year,\n",
|
||||
" livability_score,\n",
|
||||
" safety_score,\n",
|
||||
" affordability_score,\n",
|
||||
" amenity_score,\n",
|
||||
" population,\n",
|
||||
" median_household_income\n",
|
||||
"FROM public_marts.mart_neighbourhood_overview\n",
|
||||
"WHERE year = (SELECT MAX(year) FROM public_marts.mart_neighbourhood_overview)\n",
|
||||
"ORDER BY livability_score DESC\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"df = pd.read_sql(query, engine)\n",
|
||||
"print(f\"Loaded {len(df)} neighbourhoods\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Transformation Steps\n",
|
||||
"\n",
|
||||
"1. Filter to most recent year of data\n",
|
||||
"2. Extract GeoJSON from PostGIS geometry column\n",
|
||||
"3. Pass to choropleth figure factory"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Transform geometry to GeoJSON\n",
|
||||
"import json\n",
|
||||
"\n",
|
||||
"import geopandas as gpd\n",
|
||||
"\n",
|
||||
"# Convert WKB geometry to GeoDataFrame\n",
|
||||
"gdf = gpd.GeoDataFrame(\n",
|
||||
" df, geometry=gpd.GeoSeries.from_wkb(df[\"geometry\"]), crs=\"EPSG:4326\"\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Create GeoJSON FeatureCollection\n",
|
||||
"geojson = json.loads(gdf.to_json())\n",
|
||||
"\n",
|
||||
"# Prepare data for figure factory\n",
|
||||
"data = df.drop(columns=[\"geometry\"]).to_dict(\"records\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Sample Output"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df[\n",
|
||||
" [\n",
|
||||
" \"neighbourhood_name\",\n",
|
||||
" \"livability_score\",\n",
|
||||
" \"safety_score\",\n",
|
||||
" \"affordability_score\",\n",
|
||||
" \"amenity_score\",\n",
|
||||
" ]\n",
|
||||
"].head(10)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 2. Data Visualization\n",
|
||||
"\n",
|
||||
"### Figure Factory\n",
|
||||
"\n",
|
||||
"Uses `create_choropleth_figure` from `portfolio_app.figures.toronto.choropleth`.\n",
|
||||
"\n",
|
||||
"**Key Parameters:**\n",
|
||||
"- `geojson`: GeoJSON FeatureCollection with neighbourhood boundaries\n",
|
||||
"- `data`: List of dicts with neighbourhood_id and scores\n",
|
||||
"- `location_key`: 'neighbourhood_id'\n",
|
||||
"- `color_column`: 'livability_score' (or safety_score, etc.)\n",
|
||||
"- `color_scale`: 'RdYlGn' (red=low, yellow=mid, green=high)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"sys.path.insert(0, \"../..\")\n",
|
||||
"\n",
|
||||
"from portfolio_app.figures.toronto.choropleth import create_choropleth_figure\n",
|
||||
"\n",
|
||||
"fig = create_choropleth_figure(\n",
|
||||
" geojson=geojson,\n",
|
||||
" data=data,\n",
|
||||
" location_key=\"neighbourhood_id\",\n",
|
||||
" color_column=\"livability_score\",\n",
|
||||
" hover_data=[\n",
|
||||
" \"neighbourhood_name\",\n",
|
||||
" \"safety_score\",\n",
|
||||
" \"affordability_score\",\n",
|
||||
" \"amenity_score\",\n",
|
||||
" ],\n",
|
||||
" color_scale=\"RdYlGn\",\n",
|
||||
" title=\"Toronto Neighbourhood Livability Score\",\n",
|
||||
" zoom=10,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"fig.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Score Components\n",
|
||||
"\n",
|
||||
"The livability score is a weighted composite:\n",
|
||||
"\n",
|
||||
"| Component | Weight | Source |\n",
|
||||
"|-----------|--------|--------|\n",
|
||||
"| Safety | 30% | Inverse of crime rate per 100K |\n",
|
||||
"| Affordability | 40% | Inverse of rent-to-income ratio |\n",
|
||||
"| Amenities | 30% | Amenities per 1,000 residents |"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "python",
|
||||
"version": "3.11.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
Reference in New Issue
Block a user