feat: add data-platform plugin (v4.0.0)

Add new data-platform plugin for data engineering workflows with:

MCP Server (32 tools):
- pandas operations (14 tools): read_csv, read_parquet, read_json,
  to_csv, to_parquet, describe, head, tail, filter, select, groupby,
  join, list_data, drop_data
- PostgreSQL/PostGIS (10 tools): pg_connect, pg_query, pg_execute,
  pg_tables, pg_columns, pg_schemas, st_tables, st_geometry_type,
  st_srid, st_extent
- dbt integration (8 tools): dbt_parse, dbt_run, dbt_test, dbt_build,
  dbt_compile, dbt_ls, dbt_docs_generate, dbt_lineage

Plugin Features:
- Arrow IPC data_ref system for DataFrame persistence across tool calls
- Pre-execution validation for dbt with `dbt parse`
- SessionStart hook for PostgreSQL connectivity check (non-blocking)
- Hybrid configuration (system ~/.config/claude/postgres.env + project .env)
- Memory management with 100k row limit and chunking support

Commands: /initial-setup, /ingest, /profile, /schema, /explain, /lineage, /run
Agents: data-ingestion, data-analysis

Test suite: 71 tests covering config, data store, pandas, postgres, dbt tools

Addresses data workflow issues from personal-portfolio project:
- Lost data after multiple interactions (solved by Arrow IPC data_ref)
- dbt 1.9+ syntax deprecation (solved by pre-execution validation)
- Ungraceful PostgreSQL error handling (solved by SessionStart hook)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
2026-01-25 14:24:03 -05:00
parent 6a267d074b
commit 89f0354ccc
39 changed files with 5413 additions and 6 deletions

View File

@@ -6,7 +6,36 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## [Unreleased]
*Changes staged for the next release*
### Added
#### New Plugin: data-platform v1.0.0
- **pandas MCP Tools** (14 tools): DataFrame operations with Arrow IPC data_ref persistence
- `read_csv`, `read_parquet`, `read_json` - Load data with chunking support
- `to_csv`, `to_parquet` - Export to various formats
- `describe`, `head`, `tail` - Data exploration
- `filter`, `select`, `groupby`, `join` - Data transformation
- `list_data`, `drop_data` - Memory management
- **PostgreSQL MCP Tools** (10 tools): Database operations with asyncpg connection pooling
- `pg_connect`, `pg_query`, `pg_execute` - Core database operations
- `pg_tables`, `pg_columns`, `pg_schemas` - Schema exploration
- `st_tables`, `st_geometry_type`, `st_srid`, `st_extent` - PostGIS spatial support
- **dbt MCP Tools** (8 tools): Build tool wrapper with pre-execution validation
- `dbt_parse` - Pre-flight validation (catches dbt 1.9+ deprecations)
- `dbt_run`, `dbt_test`, `dbt_build` - Execution with auto-validation
- `dbt_compile`, `dbt_ls`, `dbt_docs_generate`, `dbt_lineage` - Analysis tools
- **Commands**: `/ingest`, `/profile`, `/schema`, `/explain`, `/lineage`, `/run`
- **Agents**: `data-ingestion` (loading/transformation), `data-analysis` (exploration/profiling)
- **SessionStart Hook**: Graceful PostgreSQL connection check (non-blocking warning)
- **Key Features**:
- data_ref system for DataFrame persistence across tool calls
- 100k row limit with chunking support for large datasets
- Hybrid configuration (system: `~/.config/claude/postgres.env`, project: `.env`)
- Auto-detection of dbt projects
- Arrow IPC format for efficient memory management
---