refactor: multi-dashboard structural migration
Some checks failed
CI / lint-and-test (pull_request) Has been cancelled
Some checks failed
CI / lint-and-test (pull_request) Has been cancelled
- Rename dbt project from toronto_housing to portfolio - Restructure dbt models into domain subdirectories: - shared/ for cross-domain dimensions (dim_time) - staging/toronto/, intermediate/toronto/, marts/toronto/ - Update SQLAlchemy models for raw_toronto schema - Add explicit cross-schema FK relationships for FactRentals - Namespace figure factories under figures/toronto/ - Namespace notebooks under notebooks/toronto/ - Update Makefile with domain-specific targets and env loading - Update all documentation for multi-dashboard structure This enables adding new dashboard projects (e.g., /football, /energy) without structural conflicts or naming collisions. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
200
notebooks/toronto/housing/rent_trend_line.ipynb
Normal file
200
notebooks/toronto/housing/rent_trend_line.ipynb
Normal file
@@ -0,0 +1,200 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Rent Trend Line Chart\n",
|
||||
"\n",
|
||||
"Shows 5-year rental price trends across Toronto neighbourhoods."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 1. Data Reference\n",
|
||||
"\n",
|
||||
"### Source Tables\n",
|
||||
"\n",
|
||||
"| Table | Grain | Key Columns |\n",
|
||||
"|-------|-------|-------------|\n",
|
||||
"| `mart_neighbourhood_housing` | neighbourhood × year | year, avg_rent_2bed, rent_yoy_change_pct |\n",
|
||||
"\n",
|
||||
"### SQL Query"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"import pandas as pd\n",
|
||||
"from dotenv import load_dotenv\n",
|
||||
"from sqlalchemy import create_engine\n",
|
||||
"\n",
|
||||
"# Load .env from project root\n",
|
||||
"load_dotenv(\"../../.env\")\n",
|
||||
"\n",
|
||||
"engine = create_engine(os.environ[\"DATABASE_URL\"])\n",
|
||||
"\n",
|
||||
"# City-wide average rent by year\n",
|
||||
"query = \"\"\"\n",
|
||||
"SELECT\n",
|
||||
" year,\n",
|
||||
" AVG(avg_rent_bachelor) as avg_rent_bachelor,\n",
|
||||
" AVG(avg_rent_1bed) as avg_rent_1bed,\n",
|
||||
" AVG(avg_rent_2bed) as avg_rent_2bed,\n",
|
||||
" AVG(avg_rent_3bed) as avg_rent_3bed,\n",
|
||||
" AVG(rent_yoy_change_pct) as avg_yoy_change\n",
|
||||
"FROM public_marts.mart_neighbourhood_housing\n",
|
||||
"WHERE year >= (SELECT MAX(year) - 5 FROM public_marts.mart_neighbourhood_housing)\n",
|
||||
"GROUP BY year\n",
|
||||
"ORDER BY year\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"df = pd.read_sql(query, engine)\n",
|
||||
"print(f\"Loaded {len(df)} years of rent data\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Transformation Steps\n",
|
||||
"\n",
|
||||
"1. Aggregate rent by year (city-wide average)\n",
|
||||
"2. Convert year to datetime for proper x-axis\n",
|
||||
"3. Reshape for multi-line chart by bedroom type"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Create date column from year\n",
|
||||
"df[\"date\"] = pd.to_datetime(df[\"year\"].astype(str) + \"-01-01\")\n",
|
||||
"\n",
|
||||
"# Melt for multi-line chart\n",
|
||||
"df_melted = df.melt(\n",
|
||||
" id_vars=[\"year\", \"date\"],\n",
|
||||
" value_vars=[\"avg_rent_bachelor\", \"avg_rent_1bed\", \"avg_rent_2bed\", \"avg_rent_3bed\"],\n",
|
||||
" var_name=\"bedroom_type\",\n",
|
||||
" value_name=\"avg_rent\",\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Clean labels\n",
|
||||
"df_melted[\"bedroom_type\"] = df_melted[\"bedroom_type\"].map(\n",
|
||||
" {\n",
|
||||
" \"avg_rent_bachelor\": \"Bachelor\",\n",
|
||||
" \"avg_rent_1bed\": \"1 Bedroom\",\n",
|
||||
" \"avg_rent_2bed\": \"2 Bedroom\",\n",
|
||||
" \"avg_rent_3bed\": \"3 Bedroom\",\n",
|
||||
" }\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Sample Output"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df[\n",
|
||||
" [\n",
|
||||
" \"year\",\n",
|
||||
" \"avg_rent_bachelor\",\n",
|
||||
" \"avg_rent_1bed\",\n",
|
||||
" \"avg_rent_2bed\",\n",
|
||||
" \"avg_rent_3bed\",\n",
|
||||
" \"avg_yoy_change\",\n",
|
||||
" ]\n",
|
||||
"]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 2. Data Visualization\n",
|
||||
"\n",
|
||||
"### Figure Factory\n",
|
||||
"\n",
|
||||
"Uses `create_price_time_series` from `portfolio_app.figures.toronto.time_series`.\n",
|
||||
"\n",
|
||||
"**Key Parameters:**\n",
|
||||
"- `date_column`: 'date'\n",
|
||||
"- `price_column`: 'avg_rent'\n",
|
||||
"- `group_column`: 'bedroom_type' (for multi-line)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"sys.path.insert(0, \"../..\")\n",
|
||||
"\n",
|
||||
"from portfolio_app.figures.toronto.time_series import create_price_time_series\n",
|
||||
"\n",
|
||||
"data = df_melted.to_dict(\"records\")\n",
|
||||
"\n",
|
||||
"fig = create_price_time_series(\n",
|
||||
" data=data,\n",
|
||||
" date_column=\"date\",\n",
|
||||
" price_column=\"avg_rent\",\n",
|
||||
" group_column=\"bedroom_type\",\n",
|
||||
" title=\"Toronto Average Rent Trend (5 Years)\",\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"fig.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### YoY Change Analysis"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Show year-over-year changes\n",
|
||||
"print(\"Year-over-Year Rent Change (%)\")\n",
|
||||
"df[[\"year\", \"avg_yoy_change\"]].dropna()"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "python",
|
||||
"version": "3.11.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
Reference in New Issue
Block a user