refactor: multi-dashboard structural migration
Some checks failed
CI / lint-and-test (pull_request) Has been cancelled
Some checks failed
CI / lint-and-test (pull_request) Has been cancelled
- Rename dbt project from toronto_housing to portfolio - Restructure dbt models into domain subdirectories: - shared/ for cross-domain dimensions (dim_time) - staging/toronto/, intermediate/toronto/, marts/toronto/ - Update SQLAlchemy models for raw_toronto schema - Add explicit cross-schema FK relationships for FactRentals - Namespace figure factories under figures/toronto/ - Namespace notebooks under notebooks/toronto/ - Update Makefile with domain-specific targets and env loading - Update all documentation for multi-dashboard structure This enables adding new dashboard projects (e.g., /football, /energy) without structural conflicts or naming collisions. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
200
notebooks/toronto/safety/crime_breakdown_bar.ipynb
Normal file
200
notebooks/toronto/safety/crime_breakdown_bar.ipynb
Normal file
@@ -0,0 +1,200 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Crime Type Breakdown Bar Chart\n",
|
||||
"\n",
|
||||
"Stacked bar chart showing crime composition by Major Crime Indicator (MCI) categories."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 1. Data Reference\n",
|
||||
"\n",
|
||||
"### Source Tables\n",
|
||||
"\n",
|
||||
"| Table | Grain | Key Columns |\n",
|
||||
"|-------|-------|-------------|\n",
|
||||
"| `mart_neighbourhood_safety` | neighbourhood × year | assault_count, auto_theft_count, break_enter_count, robbery_count, etc. |\n",
|
||||
"\n",
|
||||
"### SQL Query"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"import pandas as pd\n",
|
||||
"from dotenv import load_dotenv\n",
|
||||
"from sqlalchemy import create_engine\n",
|
||||
"\n",
|
||||
"# Load .env from project root\n",
|
||||
"load_dotenv(\"../../.env\")\n",
|
||||
"\n",
|
||||
"engine = create_engine(os.environ[\"DATABASE_URL\"])\n",
|
||||
"\n",
|
||||
"query = \"\"\"\n",
|
||||
"SELECT\n",
|
||||
" neighbourhood_name,\n",
|
||||
" assault_count,\n",
|
||||
" auto_theft_count,\n",
|
||||
" break_enter_count,\n",
|
||||
" robbery_count,\n",
|
||||
" theft_over_count,\n",
|
||||
" homicide_count,\n",
|
||||
" total_incidents,\n",
|
||||
" crime_rate_per_100k\n",
|
||||
"FROM public_marts.mart_neighbourhood_safety\n",
|
||||
"WHERE year = (SELECT MAX(year) FROM public_marts.mart_neighbourhood_safety)\n",
|
||||
"ORDER BY total_incidents DESC\n",
|
||||
"LIMIT 15\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"df = pd.read_sql(query, engine)\n",
|
||||
"print(f\"Loaded top {len(df)} neighbourhoods by crime volume\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Transformation Steps\n",
|
||||
"\n",
|
||||
"1. Select top 15 neighbourhoods by total incidents\n",
|
||||
"2. Melt crime type columns into rows\n",
|
||||
"3. Pass to stacked bar figure factory"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df_melted = df.melt(\n",
|
||||
" id_vars=[\"neighbourhood_name\", \"total_incidents\"],\n",
|
||||
" value_vars=[\n",
|
||||
" \"assault_count\",\n",
|
||||
" \"auto_theft_count\",\n",
|
||||
" \"break_enter_count\",\n",
|
||||
" \"robbery_count\",\n",
|
||||
" \"theft_over_count\",\n",
|
||||
" \"homicide_count\",\n",
|
||||
" ],\n",
|
||||
" var_name=\"crime_type\",\n",
|
||||
" value_name=\"count\",\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Clean labels\n",
|
||||
"df_melted[\"crime_type\"] = (\n",
|
||||
" df_melted[\"crime_type\"].str.replace(\"_count\", \"\").str.replace(\"_\", \" \").str.title()\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"data = df_melted.to_dict(\"records\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Sample Output"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df[\n",
|
||||
" [\n",
|
||||
" \"neighbourhood_name\",\n",
|
||||
" \"assault_count\",\n",
|
||||
" \"auto_theft_count\",\n",
|
||||
" \"break_enter_count\",\n",
|
||||
" \"total_incidents\",\n",
|
||||
" ]\n",
|
||||
"].head(10)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 2. Data Visualization\n",
|
||||
"\n",
|
||||
"### Figure Factory\n",
|
||||
"\n",
|
||||
"Uses `create_stacked_bar` from `portfolio_app.figures.toronto.bar_charts`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"sys.path.insert(0, \"../..\")\n",
|
||||
"\n",
|
||||
"from portfolio_app.figures.toronto.bar_charts import create_stacked_bar\n",
|
||||
"\n",
|
||||
"fig = create_stacked_bar(\n",
|
||||
" data=data,\n",
|
||||
" x_column=\"neighbourhood_name\",\n",
|
||||
" value_column=\"count\",\n",
|
||||
" category_column=\"crime_type\",\n",
|
||||
" title=\"Crime Type Breakdown - Top 15 Neighbourhoods\",\n",
|
||||
" color_map={\n",
|
||||
" \"Assault\": \"#d62728\",\n",
|
||||
" \"Auto Theft\": \"#ff7f0e\",\n",
|
||||
" \"Break Enter\": \"#9467bd\",\n",
|
||||
" \"Robbery\": \"#8c564b\",\n",
|
||||
" \"Theft Over\": \"#e377c2\",\n",
|
||||
" \"Homicide\": \"#1f77b4\",\n",
|
||||
" },\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"fig.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### MCI Categories\n",
|
||||
"\n",
|
||||
"| Category | Description |\n",
|
||||
"|----------|------------|\n",
|
||||
"| Assault | Physical attacks |\n",
|
||||
"| Auto Theft | Vehicle theft |\n",
|
||||
"| Break & Enter | Burglary |\n",
|
||||
"| Robbery | Theft with force/threat |\n",
|
||||
"| Theft Over | Theft > $5,000 |\n",
|
||||
"| Homicide | Murder/manslaughter |"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "python",
|
||||
"version": "3.11.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
Reference in New Issue
Block a user