docs: Complete Phase 6 notebooks and Phase 7 documentation review
Phase 6 - Jupyter Notebooks (15 total): - Overview tab: livability_choropleth, top_bottom_10_bar, income_safety_scatter - Housing tab: affordability_choropleth, rent_trend_line, tenure_breakdown_bar - Safety tab: crime_rate_choropleth, crime_breakdown_bar, crime_trend_line - Demographics tab: income_choropleth, age_distribution, population_density_bar - Amenities tab: amenity_index_choropleth, amenity_radar, transit_accessibility_bar Phase 7 - Documentation: - Updated CLAUDE.md with Sprint 9 completion status - Added notebooks directory to application structure - Expanded figures directory listing Closes #71, #72, #73, #74, #75, #76, #77 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
174
notebooks/housing/affordability_choropleth.ipynb
Normal file
174
notebooks/housing/affordability_choropleth.ipynb
Normal file
@@ -0,0 +1,174 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Affordability Index Choropleth Map\n",
|
||||
"\n",
|
||||
"Displays housing affordability across Toronto's 158 neighbourhoods. Index of 100 = city average."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 1. Data Reference\n",
|
||||
"\n",
|
||||
"### Source Tables\n",
|
||||
"\n",
|
||||
"| Table | Grain | Key Columns |\n",
|
||||
"|-------|-------|-------------|\n",
|
||||
"| `mart_neighbourhood_housing` | neighbourhood × year | affordability_index, rent_to_income_pct, avg_rent_2bed, geometry |\n",
|
||||
"\n",
|
||||
"### SQL Query"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import pandas as pd\n",
|
||||
"from sqlalchemy import create_engine\n",
|
||||
"import os\n",
|
||||
"\n",
|
||||
"engine = create_engine(os.environ.get('DATABASE_URL', 'postgresql://portfolio:portfolio@localhost:5432/portfolio'))\n",
|
||||
"\n",
|
||||
"query = \"\"\"\n",
|
||||
"SELECT\n",
|
||||
" neighbourhood_id,\n",
|
||||
" neighbourhood_name,\n",
|
||||
" geometry,\n",
|
||||
" year,\n",
|
||||
" affordability_index,\n",
|
||||
" rent_to_income_pct,\n",
|
||||
" avg_rent_2bed,\n",
|
||||
" median_household_income,\n",
|
||||
" is_affordable\n",
|
||||
"FROM mart_neighbourhood_housing\n",
|
||||
"WHERE year = (SELECT MAX(year) FROM mart_neighbourhood_housing)\n",
|
||||
"ORDER BY affordability_index ASC\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"df = pd.read_sql(query, engine)\n",
|
||||
"print(f\"Loaded {len(df)} neighbourhoods\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Transformation Steps\n",
|
||||
"\n",
|
||||
"1. Filter to most recent year\n",
|
||||
"2. Convert geometry to GeoJSON\n",
|
||||
"3. Lower index = more affordable (inverted for visualization clarity)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import geopandas as gpd\n",
|
||||
"import json\n",
|
||||
"\n",
|
||||
"gdf = gpd.GeoDataFrame(\n",
|
||||
" df,\n",
|
||||
" geometry=gpd.GeoSeries.from_wkb(df['geometry']),\n",
|
||||
" crs='EPSG:4326'\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"geojson = json.loads(gdf.to_json())\n",
|
||||
"data = df.drop(columns=['geometry']).to_dict('records')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Sample Output"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df[['neighbourhood_name', 'affordability_index', 'rent_to_income_pct', 'avg_rent_2bed', 'is_affordable']].head(10)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 2. Data Visualization\n",
|
||||
"\n",
|
||||
"### Figure Factory\n",
|
||||
"\n",
|
||||
"Uses `create_choropleth_figure` from `portfolio_app.figures.choropleth`.\n",
|
||||
"\n",
|
||||
"**Key Parameters:**\n",
|
||||
"- `color_column`: 'affordability_index'\n",
|
||||
"- `color_scale`: 'RdYlGn_r' (reversed: green=affordable, red=expensive)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"sys.path.insert(0, '../..')\n",
|
||||
"\n",
|
||||
"from portfolio_app.figures.choropleth import create_choropleth_figure\n",
|
||||
"\n",
|
||||
"fig = create_choropleth_figure(\n",
|
||||
" geojson=geojson,\n",
|
||||
" data=data,\n",
|
||||
" location_key='neighbourhood_id',\n",
|
||||
" color_column='affordability_index',\n",
|
||||
" hover_data=['neighbourhood_name', 'rent_to_income_pct', 'avg_rent_2bed'],\n",
|
||||
" color_scale='RdYlGn_r', # Reversed: lower index (affordable) = green\n",
|
||||
" title='Toronto Housing Affordability Index',\n",
|
||||
" zoom=10,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"fig.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Index Interpretation\n",
|
||||
"\n",
|
||||
"| Index | Meaning |\n",
|
||||
"|-------|--------|\n",
|
||||
"| < 100 | More affordable than city average |\n",
|
||||
"| = 100 | City average affordability |\n",
|
||||
"| > 100 | Less affordable than city average |\n",
|
||||
"\n",
|
||||
"Affordability calculated as: `rent_to_income_pct / city_avg_rent_to_income * 100`"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "python",
|
||||
"version": "3.11.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
183
notebooks/housing/rent_trend_line.ipynb
Normal file
183
notebooks/housing/rent_trend_line.ipynb
Normal file
@@ -0,0 +1,183 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Rent Trend Line Chart\n",
|
||||
"\n",
|
||||
"Shows 5-year rental price trends across Toronto neighbourhoods."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 1. Data Reference\n",
|
||||
"\n",
|
||||
"### Source Tables\n",
|
||||
"\n",
|
||||
"| Table | Grain | Key Columns |\n",
|
||||
"|-------|-------|-------------|\n",
|
||||
"| `mart_neighbourhood_housing` | neighbourhood × year | year, avg_rent_2bed, rent_yoy_change_pct |\n",
|
||||
"\n",
|
||||
"### SQL Query"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import pandas as pd\n",
|
||||
"from sqlalchemy import create_engine\n",
|
||||
"import os\n",
|
||||
"\n",
|
||||
"engine = create_engine(os.environ.get('DATABASE_URL', 'postgresql://portfolio:portfolio@localhost:5432/portfolio'))\n",
|
||||
"\n",
|
||||
"# City-wide average rent by year\n",
|
||||
"query = \"\"\"\n",
|
||||
"SELECT\n",
|
||||
" year,\n",
|
||||
" AVG(avg_rent_bachelor) as avg_rent_bachelor,\n",
|
||||
" AVG(avg_rent_1bed) as avg_rent_1bed,\n",
|
||||
" AVG(avg_rent_2bed) as avg_rent_2bed,\n",
|
||||
" AVG(avg_rent_3bed) as avg_rent_3bed,\n",
|
||||
" AVG(rent_yoy_change_pct) as avg_yoy_change\n",
|
||||
"FROM mart_neighbourhood_housing\n",
|
||||
"WHERE year >= (SELECT MAX(year) - 5 FROM mart_neighbourhood_housing)\n",
|
||||
"GROUP BY year\n",
|
||||
"ORDER BY year\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"df = pd.read_sql(query, engine)\n",
|
||||
"print(f\"Loaded {len(df)} years of rent data\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Transformation Steps\n",
|
||||
"\n",
|
||||
"1. Aggregate rent by year (city-wide average)\n",
|
||||
"2. Convert year to datetime for proper x-axis\n",
|
||||
"3. Reshape for multi-line chart by bedroom type"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Create date column from year\n",
|
||||
"df['date'] = pd.to_datetime(df['year'].astype(str) + '-01-01')\n",
|
||||
"\n",
|
||||
"# Melt for multi-line chart\n",
|
||||
"df_melted = df.melt(\n",
|
||||
" id_vars=['year', 'date'],\n",
|
||||
" value_vars=['avg_rent_bachelor', 'avg_rent_1bed', 'avg_rent_2bed', 'avg_rent_3bed'],\n",
|
||||
" var_name='bedroom_type',\n",
|
||||
" value_name='avg_rent'\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Clean labels\n",
|
||||
"df_melted['bedroom_type'] = df_melted['bedroom_type'].map({\n",
|
||||
" 'avg_rent_bachelor': 'Bachelor',\n",
|
||||
" 'avg_rent_1bed': '1 Bedroom',\n",
|
||||
" 'avg_rent_2bed': '2 Bedroom',\n",
|
||||
" 'avg_rent_3bed': '3 Bedroom'\n",
|
||||
"})"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Sample Output"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df[['year', 'avg_rent_bachelor', 'avg_rent_1bed', 'avg_rent_2bed', 'avg_rent_3bed', 'avg_yoy_change']]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 2. Data Visualization\n",
|
||||
"\n",
|
||||
"### Figure Factory\n",
|
||||
"\n",
|
||||
"Uses `create_price_time_series` from `portfolio_app.figures.time_series`.\n",
|
||||
"\n",
|
||||
"**Key Parameters:**\n",
|
||||
"- `date_column`: 'date'\n",
|
||||
"- `price_column`: 'avg_rent'\n",
|
||||
"- `group_column`: 'bedroom_type' (for multi-line)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"sys.path.insert(0, '../..')\n",
|
||||
"\n",
|
||||
"from portfolio_app.figures.time_series import create_price_time_series\n",
|
||||
"\n",
|
||||
"data = df_melted.to_dict('records')\n",
|
||||
"\n",
|
||||
"fig = create_price_time_series(\n",
|
||||
" data=data,\n",
|
||||
" date_column='date',\n",
|
||||
" price_column='avg_rent',\n",
|
||||
" group_column='bedroom_type',\n",
|
||||
" title='Toronto Average Rent Trend (5 Years)',\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"fig.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### YoY Change Analysis"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Show year-over-year changes\n",
|
||||
"print(\"Year-over-Year Rent Change (%)\")\n",
|
||||
"df[['year', 'avg_yoy_change']].dropna()"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "python",
|
||||
"version": "3.11.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
188
notebooks/housing/tenure_breakdown_bar.ipynb
Normal file
188
notebooks/housing/tenure_breakdown_bar.ipynb
Normal file
@@ -0,0 +1,188 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Housing Tenure Breakdown Bar Chart\n",
|
||||
"\n",
|
||||
"Shows the distribution of owner-occupied vs renter-occupied dwellings across neighbourhoods."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 1. Data Reference\n",
|
||||
"\n",
|
||||
"### Source Tables\n",
|
||||
"\n",
|
||||
"| Table | Grain | Key Columns |\n",
|
||||
"|-------|-------|-------------|\n",
|
||||
"| `mart_neighbourhood_housing` | neighbourhood × year | pct_owner_occupied, pct_renter_occupied, income_quintile |\n",
|
||||
"\n",
|
||||
"### SQL Query"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import pandas as pd\n",
|
||||
"from sqlalchemy import create_engine\n",
|
||||
"import os\n",
|
||||
"\n",
|
||||
"engine = create_engine(os.environ.get('DATABASE_URL', 'postgresql://portfolio:portfolio@localhost:5432/portfolio'))\n",
|
||||
"\n",
|
||||
"query = \"\"\"\n",
|
||||
"SELECT\n",
|
||||
" neighbourhood_name,\n",
|
||||
" pct_owner_occupied,\n",
|
||||
" pct_renter_occupied,\n",
|
||||
" income_quintile,\n",
|
||||
" total_rental_units,\n",
|
||||
" average_dwelling_value\n",
|
||||
"FROM mart_neighbourhood_housing\n",
|
||||
"WHERE year = (SELECT MAX(year) FROM mart_neighbourhood_housing)\n",
|
||||
" AND pct_owner_occupied IS NOT NULL\n",
|
||||
"ORDER BY pct_renter_occupied DESC\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"df = pd.read_sql(query, engine)\n",
|
||||
"print(f\"Loaded {len(df)} neighbourhoods with tenure data\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Transformation Steps\n",
|
||||
"\n",
|
||||
"1. Filter to most recent year with tenure data\n",
|
||||
"2. Melt owner/renter columns for stacked bar\n",
|
||||
"3. Sort by renter percentage (highest first)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prepare for stacked bar\n",
|
||||
"df_stacked = df.melt(\n",
|
||||
" id_vars=['neighbourhood_name', 'income_quintile'],\n",
|
||||
" value_vars=['pct_owner_occupied', 'pct_renter_occupied'],\n",
|
||||
" var_name='tenure_type',\n",
|
||||
" value_name='percentage'\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"df_stacked['tenure_type'] = df_stacked['tenure_type'].map({\n",
|
||||
" 'pct_owner_occupied': 'Owner',\n",
|
||||
" 'pct_renter_occupied': 'Renter'\n",
|
||||
"})\n",
|
||||
"\n",
|
||||
"data = df_stacked.to_dict('records')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Sample Output"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(\"Highest Renter Neighbourhoods:\")\n",
|
||||
"df[['neighbourhood_name', 'pct_renter_occupied', 'pct_owner_occupied', 'income_quintile']].head(10)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 2. Data Visualization\n",
|
||||
"\n",
|
||||
"### Figure Factory\n",
|
||||
"\n",
|
||||
"Uses `create_stacked_bar` from `portfolio_app.figures.bar_charts`.\n",
|
||||
"\n",
|
||||
"**Key Parameters:**\n",
|
||||
"- `x_column`: 'neighbourhood_name'\n",
|
||||
"- `value_column`: 'percentage'\n",
|
||||
"- `category_column`: 'tenure_type'\n",
|
||||
"- `show_percentages`: True"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"sys.path.insert(0, '../..')\n",
|
||||
"\n",
|
||||
"from portfolio_app.figures.bar_charts import create_stacked_bar\n",
|
||||
"\n",
|
||||
"# Show top 20 by renter percentage\n",
|
||||
"top_20_names = df.head(20)['neighbourhood_name'].tolist()\n",
|
||||
"data_filtered = [d for d in data if d['neighbourhood_name'] in top_20_names]\n",
|
||||
"\n",
|
||||
"fig = create_stacked_bar(\n",
|
||||
" data=data_filtered,\n",
|
||||
" x_column='neighbourhood_name',\n",
|
||||
" value_column='percentage',\n",
|
||||
" category_column='tenure_type',\n",
|
||||
" title='Housing Tenure Mix - Top 20 Renter Neighbourhoods',\n",
|
||||
" color_map={'Owner': '#4CAF50', 'Renter': '#2196F3'},\n",
|
||||
" show_percentages=True,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"fig.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### City-Wide Distribution"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# City-wide averages\n",
|
||||
"print(f\"City Average Owner-Occupied: {df['pct_owner_occupied'].mean():.1f}%\")\n",
|
||||
"print(f\"City Average Renter-Occupied: {df['pct_renter_occupied'].mean():.1f}%\")\n",
|
||||
"\n",
|
||||
"# By income quintile\n",
|
||||
"print(\"\\nTenure by Income Quintile:\")\n",
|
||||
"df.groupby('income_quintile')[['pct_owner_occupied', 'pct_renter_occupied']].mean().round(1)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "python",
|
||||
"version": "3.11.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
Reference in New Issue
Block a user