From cda2a078d9478068a0964099b958b83813219b96 Mon Sep 17 00:00:00 2001 From: l3ocho Date: Mon, 2 Feb 2026 12:32:39 -0500 Subject: [PATCH] refactor(dbt): migrate to domain-scoped schema names MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Create generate_schema_name macro to use custom schema names directly - Update dbt_project.yml schemas: staging→stg_toronto, intermediate→int_toronto, marts→mart_toronto - Add dbt/macros/toronto/ directory for future domain-specific macros - Fix documentation drift in PROJECT_REFERENCE.md (load-data-only→load-toronto-only) - Update DATABASE_SCHEMA.md with new schema names - Update CLAUDE.md database schemas table - Update adding-dashboard.md runbook with domain-scoped pattern Co-Authored-By: Claude Opus 4.5 --- CLAUDE.md | 6 +++--- dbt/dbt_project.yml | 6 +++--- dbt/macros/generate_schema_name.sql | 11 +++++++++++ dbt/macros/toronto/.gitkeep | 0 docs/DATABASE_SCHEMA.md | 10 +++++----- docs/PROJECT_REFERENCE.md | 2 +- docs/runbooks/adding-dashboard.md | 10 +++++++++- 7 files changed, 32 insertions(+), 13 deletions(-) create mode 100644 dbt/macros/generate_schema_name.sql create mode 100644 dbt/macros/toronto/.gitkeep diff --git a/CLAUDE.md b/CLAUDE.md index ca0d430..67f3d62 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -208,9 +208,9 @@ The codebase is structured to support multiple dashboard projects: |--------|---------| | `public` | Shared dimensions (dim_time) | | `raw_toronto` | Toronto-specific raw/dimension tables | -| `staging` | dbt staging views | -| `intermediate` | dbt intermediate views | -| `marts` | dbt mart tables | +| `stg_toronto` | Toronto dbt staging views | +| `int_toronto` | Toronto dbt intermediate views | +| `mart_toronto` | Toronto dbt mart tables | ### Geographic Reality (Toronto Housing) diff --git a/dbt/dbt_project.yml b/dbt/dbt_project.yml index 0baa6f7..16187aa 100644 --- a/dbt/dbt_project.yml +++ b/dbt/dbt_project.yml @@ -22,12 +22,12 @@ models: staging: toronto: +materialized: view - +schema: staging + +schema: stg_toronto intermediate: toronto: +materialized: view - +schema: intermediate + +schema: int_toronto marts: toronto: +materialized: table - +schema: marts + +schema: mart_toronto diff --git a/dbt/macros/generate_schema_name.sql b/dbt/macros/generate_schema_name.sql new file mode 100644 index 0000000..1cedfa5 --- /dev/null +++ b/dbt/macros/generate_schema_name.sql @@ -0,0 +1,11 @@ +-- Override dbt default schema name generation. +-- Use the custom schema name directly instead of +-- concatenating with the target schema. +-- See: https://docs.getdbt.com/docs/build/custom-schemas +{% macro generate_schema_name(custom_schema_name, node) %} + {%- if custom_schema_name is none -%} + {{ target.schema }} + {%- else -%} + {{ custom_schema_name | trim }} + {%- endif -%} +{% endmacro %} diff --git a/dbt/macros/toronto/.gitkeep b/dbt/macros/toronto/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/docs/DATABASE_SCHEMA.md b/docs/DATABASE_SCHEMA.md index c94a578..02bf256 100644 --- a/docs/DATABASE_SCHEMA.md +++ b/docs/DATABASE_SCHEMA.md @@ -122,9 +122,9 @@ erDiagram |--------|---------|------------| | `public` | Shared dimensions (dim_time) | SQLAlchemy | | `raw_toronto` | Toronto dimension and fact tables | SQLAlchemy | -| `staging` | Staging models | dbt | -| `intermediate` | Intermediate models | dbt | -| `marts` | Analytical tables | dbt | +| `stg_toronto` | Toronto staging models | dbt | +| `int_toronto` | Toronto intermediate models | dbt | +| `mart_toronto` | Toronto analytical tables | dbt | ### Raw Toronto Schema (raw_toronto) @@ -149,7 +149,7 @@ Shared dimensions used across all projects: |-------|-------------| | `dim_time` | Time dimension (monthly grain) | -### Staging Schema (dbt) +### Staging Schema - stg_toronto (dbt) Staging models provide 1:1 cleaned representations of source data: @@ -164,7 +164,7 @@ Staging models provide 1:1 cleaned representations of source data: | `stg_dimensions__cmhc_zones` | raw.cmhc_zones | CMHC zone boundaries | | `stg_cmhc__zone_crosswalk` | raw.crosswalk | Zone-neighbourhood mapping | -### Marts Schema (dbt) +### Marts Schema - mart_toronto (dbt) Analytical tables ready for dashboard consumption: diff --git a/docs/PROJECT_REFERENCE.md b/docs/PROJECT_REFERENCE.md index cf0e732..c76ade4 100644 --- a/docs/PROJECT_REFERENCE.md +++ b/docs/PROJECT_REFERENCE.md @@ -269,7 +269,7 @@ LOG_LEVEL=INFO | `db-init` | Initialize database schema | | `db-reset` | Drop and recreate database (DESTRUCTIVE) | | `load-data` | Load Toronto data from APIs, seed dev data | -| `load-data-only` | Load Toronto data without dbt or seeding | +| `load-toronto-only` | Load Toronto data without dbt or seeding | | `seed-data` | Seed sample development data | | `run` | Start Dash dev server | | `test` | Run pytest | diff --git a/docs/runbooks/adding-dashboard.md b/docs/runbooks/adding-dashboard.md index a0ff4d8..cfba073 100644 --- a/docs/runbooks/adding-dashboard.md +++ b/docs/runbooks/adding-dashboard.md @@ -103,7 +103,15 @@ models: staging: {dashboard_name}: +materialized: view - +schema: staging + +schema: stg_{dashboard_name} + intermediate: + {dashboard_name}: + +materialized: view + +schema: int_{dashboard_name} + marts: + {dashboard_name}: + +materialized: table + +schema: mart_{dashboard_name} ``` Follow naming conventions: