Create neighbourhood and census loaders #57

New Issue

lmiranda · 2026-01-16T15:49:51Z

lmiranda commented

2026-01-16 15:49:51 +00:00

Overview

Create database loaders for neighbourhood boundaries and census data.

Files to Create

`portfolio_app/toronto/loaders/neighbourhoods.py`

def load_neighbourhoods(
    records: list[NeighbourhoodRecord],
    session: Session | None = None,
) -> int:
    """Load neighbourhood records to dim_neighbourhood, returning count loaded."""

`portfolio_app/toronto/loaders/census.py`

def load_census_data(
    records: list[CensusRecord],
    session: Session | None = None,
) -> int:
    """Load census records to fact_census, returning count loaded."""

Acceptance Criteria

Loaders follow existing patterns in loaders/dimensions.py
Use upsert_by_key for idempotent loading
Support optional session parameter
Return count of records loaded
Import test passes
Linter passes

Technical Notes

Use existing get_session() and upsert_by_key() from loaders/base.py
Neighbourhood loader updates dim_neighbourhood table
Census loader creates records in fact_census table

Labels

Type/Feature, Priority/Medium, Component/Backend, Component/Database, Tech/Python

## Overview Create database loaders for neighbourhood boundaries and census data. ## Files to Create ### `portfolio_app/toronto/loaders/neighbourhoods.py` ```python def load_neighbourhoods( records: list[NeighbourhoodRecord], session: Session | None = None, ) -> int: """Load neighbourhood records to dim_neighbourhood, returning count loaded.""" ``` ### `portfolio_app/toronto/loaders/census.py` ```python def load_census_data( records: list[CensusRecord], session: Session | None = None, ) -> int: """Load census records to fact_census, returning count loaded.""" ``` ## Acceptance Criteria - [ ] Loaders follow existing patterns in `loaders/dimensions.py` - [ ] Use upsert_by_key for idempotent loading - [ ] Support optional session parameter - [ ] Return count of records loaded - [ ] Import test passes - [ ] Linter passes ## Technical Notes - Use existing `get_session()` and `upsert_by_key()` from `loaders/base.py` - Neighbourhood loader updates `dim_neighbourhood` table - Census loader creates records in `fact_census` table ## Labels `Type/Feature`, `Priority/Medium`, `Component/Backend`, `Component/Database`, `Tech/Python`

lmiranda commented

2026-01-16 16:01:18 +00:00

Starting implementation of neighbourhood and census loaders.

lmiranda commented

2026-01-16 16:03:20 +00:00

Implementation complete:

Created loaders/census.py with load_census_data() function
Uses upsert by (neighbourhood_id, census_year) key
Import test passed

Note: Neighbourhood loader already exists in loaders/dimensions.py. The existing load_neighbourhoods() function handles the Neighbourhood schema. For NeighbourhoodRecord from the API parser, a conversion step may be needed.

Implementation complete: - Created `loaders/census.py` with `load_census_data()` function - Uses upsert by (neighbourhood_id, census_year) key - Import test passed Note: Neighbourhood loader already exists in `loaders/dimensions.py`. The existing `load_neighbourhoods()` function handles the `Neighbourhood` schema. For `NeighbourhoodRecord` from the API parser, a conversion step may be needed.

lmiranda closed this issue

2026-01-16 16:03:26 +00:00

lmiranda added this to the Launch: Host, Bio and Toronto House Market Analysis project 2026-01-16 16:06:20 +00:00

lmiranda self-assigned this 2026-01-16 16:06:30 +00:00

lmiranda moved this to Done in Launch: Host, Bio and Toronto House Market Analysis on 2026-01-16 16:06:50 +00:00

lmiranda referenced this issue from a commit

2026-01-16 16:55:12 +00:00

feat: Implement Phase 3 neighbourhood data model

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: personal-projects/personal-portfolio#57