Merge pull request 'development' (#99) from development into main #102
@@ -208,9 +208,9 @@ The codebase is structured to support multiple dashboard projects:
|
||||
|--------|---------|
|
||||
| `public` | Shared dimensions (dim_time) |
|
||||
| `raw_toronto` | Toronto-specific raw/dimension tables |
|
||||
| `staging` | dbt staging views |
|
||||
| `intermediate` | dbt intermediate views |
|
||||
| `marts` | dbt mart tables |
|
||||
| `stg_toronto` | Toronto dbt staging views |
|
||||
| `int_toronto` | Toronto dbt intermediate views |
|
||||
| `mart_toronto` | Toronto dbt mart tables |
|
||||
|
||||
### Geographic Reality (Toronto Housing)
|
||||
|
||||
|
||||
@@ -22,12 +22,12 @@ models:
|
||||
staging:
|
||||
toronto:
|
||||
+materialized: view
|
||||
+schema: staging
|
||||
+schema: stg_toronto
|
||||
intermediate:
|
||||
toronto:
|
||||
+materialized: view
|
||||
+schema: intermediate
|
||||
+schema: int_toronto
|
||||
marts:
|
||||
toronto:
|
||||
+materialized: table
|
||||
+schema: marts
|
||||
+schema: mart_toronto
|
||||
|
||||
11
dbt/macros/generate_schema_name.sql
Normal file
11
dbt/macros/generate_schema_name.sql
Normal file
@@ -0,0 +1,11 @@
|
||||
-- Override dbt default schema name generation.
|
||||
-- Use the custom schema name directly instead of
|
||||
-- concatenating with the target schema.
|
||||
-- See: https://docs.getdbt.com/docs/build/custom-schemas
|
||||
{% macro generate_schema_name(custom_schema_name, node) %}
|
||||
{%- if custom_schema_name is none -%}
|
||||
{{ target.schema }}
|
||||
{%- else -%}
|
||||
{{ custom_schema_name | trim }}
|
||||
{%- endif -%}
|
||||
{% endmacro %}
|
||||
0
dbt/macros/toronto/.gitkeep
Normal file
0
dbt/macros/toronto/.gitkeep
Normal file
@@ -122,9 +122,9 @@ erDiagram
|
||||
|--------|---------|------------|
|
||||
| `public` | Shared dimensions (dim_time) | SQLAlchemy |
|
||||
| `raw_toronto` | Toronto dimension and fact tables | SQLAlchemy |
|
||||
| `staging` | Staging models | dbt |
|
||||
| `intermediate` | Intermediate models | dbt |
|
||||
| `marts` | Analytical tables | dbt |
|
||||
| `stg_toronto` | Toronto staging models | dbt |
|
||||
| `int_toronto` | Toronto intermediate models | dbt |
|
||||
| `mart_toronto` | Toronto analytical tables | dbt |
|
||||
|
||||
### Raw Toronto Schema (raw_toronto)
|
||||
|
||||
@@ -149,7 +149,7 @@ Shared dimensions used across all projects:
|
||||
|-------|-------------|
|
||||
| `dim_time` | Time dimension (monthly grain) |
|
||||
|
||||
### Staging Schema (dbt)
|
||||
### Staging Schema - stg_toronto (dbt)
|
||||
|
||||
Staging models provide 1:1 cleaned representations of source data:
|
||||
|
||||
@@ -164,7 +164,7 @@ Staging models provide 1:1 cleaned representations of source data:
|
||||
| `stg_dimensions__cmhc_zones` | raw.cmhc_zones | CMHC zone boundaries |
|
||||
| `stg_cmhc__zone_crosswalk` | raw.crosswalk | Zone-neighbourhood mapping |
|
||||
|
||||
### Marts Schema (dbt)
|
||||
### Marts Schema - mart_toronto (dbt)
|
||||
|
||||
Analytical tables ready for dashboard consumption:
|
||||
|
||||
|
||||
@@ -269,7 +269,7 @@ LOG_LEVEL=INFO
|
||||
| `db-init` | Initialize database schema |
|
||||
| `db-reset` | Drop and recreate database (DESTRUCTIVE) |
|
||||
| `load-data` | Load Toronto data from APIs, seed dev data |
|
||||
| `load-data-only` | Load Toronto data without dbt or seeding |
|
||||
| `load-toronto-only` | Load Toronto data without dbt or seeding |
|
||||
| `seed-data` | Seed sample development data |
|
||||
| `run` | Start Dash dev server |
|
||||
| `test` | Run pytest |
|
||||
|
||||
@@ -103,7 +103,15 @@ models:
|
||||
staging:
|
||||
{dashboard_name}:
|
||||
+materialized: view
|
||||
+schema: staging
|
||||
+schema: stg_{dashboard_name}
|
||||
intermediate:
|
||||
{dashboard_name}:
|
||||
+materialized: view
|
||||
+schema: int_{dashboard_name}
|
||||
marts:
|
||||
{dashboard_name}:
|
||||
+materialized: table
|
||||
+schema: mart_{dashboard_name}
|
||||
```
|
||||
|
||||
Follow naming conventions:
|
||||
|
||||
Reference in New Issue
Block a user