lmiranda
89f0354ccc
Add new data-platform plugin for data engineering workflows with: MCP Server (32 tools): - pandas operations (14 tools): read_csv, read_parquet, read_json, to_csv, to_parquet, describe, head, tail, filter, select, groupby, join, list_data, drop_data - PostgreSQL/PostGIS (10 tools): pg_connect, pg_query, pg_execute, pg_tables, pg_columns, pg_schemas, st_tables, st_geometry_type, st_srid, st_extent - dbt integration (8 tools): dbt_parse, dbt_run, dbt_test, dbt_build, dbt_compile, dbt_ls, dbt_docs_generate, dbt_lineage Plugin Features: - Arrow IPC data_ref system for DataFrame persistence across tool calls - Pre-execution validation for dbt with `dbt parse` - SessionStart hook for PostgreSQL connectivity check (non-blocking) - Hybrid configuration (system ~/.config/claude/postgres.env + project .env) - Memory management with 100k row limit and chunking support Commands: /initial-setup, /ingest, /profile, /schema, /explain, /lineage, /run Agents: data-ingestion, data-analysis Test suite: 71 tests covering config, data store, pandas, postgres, dbt tools Addresses data workflow issues from personal-portfolio project: - Lost data after multiple interactions (solved by Arrow IPC data_ref) - dbt 1.9+ syntax deprecation (solved by pre-execution validation) - Ungraceful PostgreSQL error handling (solved by SessionStart hook) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1.2 KiB
1.2 KiB
/lineage - Data Lineage Visualization
Show data lineage for dbt models or database tables.
Usage
/lineage <model_name> [--depth N]
Workflow
-
Get lineage data:
- Use
dbt_lineagefor dbt models - For database tables, trace through dbt manifest
- Use
-
Build lineage graph:
- Identify all upstream sources
- Identify all downstream consumers
- Note materialization at each node
-
Visualize:
- ASCII art dependency tree
- List format with indentation
- Show depth levels
-
Report:
- Full dependency chain
- Critical path identification
- Refresh implications
Examples
/lineage dim_customers
/lineage fct_orders --depth 3
Output Format
Sources:
└── raw_customers (source)
└── raw_orders (source)
dim_customers (table)
├── upstream:
│ └── stg_customers (view)
│ └── raw_customers (source)
└── downstream:
└── fct_orders (incremental)
└── rpt_customer_lifetime (table)
Available Tools
Use these MCP tools:
dbt_lineage- Get model dependenciesdbt_ls- List dbt resourcesdbt_docs_generate- Generate full manifest