fix: Repair data pipeline with StatCan CMHC rental data
- Add StatCan CMHC parser to fetch rental data from Statistics Canada API - Create year spine (2014-2025) as time dimension driver instead of census - Add CMA-level rental and income intermediate models - Update mart_neighbourhood_overview to use rental years as base - Fix neighbourhood_service queries to match dbt schema - Add CMHC data loading to pipeline script Data now flows correctly: 158 neighbourhoods × 12 years = 1,896 records Rent data available 2019-2025, crime data 2014-2024 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -64,15 +64,17 @@ crime_summary as (
|
||||
w.robbery_count,
|
||||
w.theft_over_count,
|
||||
w.homicide_count,
|
||||
w.avg_rate_per_100k,
|
||||
w.yoy_change_pct,
|
||||
|
||||
-- Crime rate per 100K population
|
||||
case
|
||||
when n.population > 0
|
||||
then round(w.total_incidents::numeric / n.population * 100000, 2)
|
||||
else null
|
||||
end as crime_rate_per_100k
|
||||
-- Crime rate per 100K population (use source data avg, or calculate if population available)
|
||||
coalesce(
|
||||
w.avg_rate_per_100k,
|
||||
case
|
||||
when n.population > 0
|
||||
then round(w.total_incidents::numeric / n.population * 100000, 2)
|
||||
else null
|
||||
end
|
||||
) as crime_rate_per_100k
|
||||
|
||||
from neighbourhoods n
|
||||
inner join with_yoy w on n.neighbourhood_id = w.neighbourhood_id
|
||||
|
||||
Reference in New Issue
Block a user