feat: add data-platform plugin (v4.0.0)

Add new data-platform plugin for data engineering workflows with:

MCP Server (32 tools):
- pandas operations (14 tools): read_csv, read_parquet, read_json,
  to_csv, to_parquet, describe, head, tail, filter, select, groupby,
  join, list_data, drop_data
- PostgreSQL/PostGIS (10 tools): pg_connect, pg_query, pg_execute,
  pg_tables, pg_columns, pg_schemas, st_tables, st_geometry_type,
  st_srid, st_extent
- dbt integration (8 tools): dbt_parse, dbt_run, dbt_test, dbt_build,
  dbt_compile, dbt_ls, dbt_docs_generate, dbt_lineage

Plugin Features:
- Arrow IPC data_ref system for DataFrame persistence across tool calls
- Pre-execution validation for dbt with `dbt parse`
- SessionStart hook for PostgreSQL connectivity check (non-blocking)
- Hybrid configuration (system ~/.config/claude/postgres.env + project .env)
- Memory management with 100k row limit and chunking support

Commands: /initial-setup, /ingest, /profile, /schema, /explain, /lineage, /run
Agents: data-ingestion, data-analysis

Test suite: 71 tests covering config, data store, pandas, postgres, dbt tools

Addresses data workflow issues from personal-portfolio project:
- Lost data after multiple interactions (solved by Arrow IPC data_ref)
- dbt 1.9+ syntax deprecation (solved by pre-execution validation)
- Ungraceful PostgreSQL error handling (solved by SessionStart hook)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
2026-01-25 14:24:03 -05:00
parent 6a267d074b
commit 89f0354ccc
39 changed files with 5413 additions and 6 deletions

View File

@@ -116,6 +116,15 @@ verify_symlinks() {
log_error "pr-review/gitea symlink missing"
log_todo "Run: ln -s ../../../mcp-servers/gitea plugins/pr-review/mcp-servers/gitea"
fi
# Check data-platform -> data-platform symlink
local dataplatform_link="$REPO_ROOT/plugins/data-platform/mcp-servers/data-platform"
if [[ -L "$dataplatform_link" ]]; then
log_success "data-platform symlink exists"
else
log_error "data-platform symlink missing"
log_todo "Run: ln -s ../../../mcp-servers/data-platform plugins/data-platform/mcp-servers/data-platform"
fi
}
# --- Section 3: Config File Templates ---
@@ -178,6 +187,22 @@ EOF
chmod 600 "$config_dir/git-flow.env"
log_success "git-flow.env template created"
fi
# PostgreSQL config (for data-platform, optional)
if [[ -f "$config_dir/postgres.env" ]]; then
log_skip "postgres.env already exists"
else
cat > "$config_dir/postgres.env" << 'EOF'
# PostgreSQL Configuration (for data-platform plugin)
# Update with your PostgreSQL connection URL
# This is OPTIONAL - pandas tools work without it
POSTGRES_URL=postgresql://user:password@localhost:5432/database
EOF
chmod 600 "$config_dir/postgres.env"
log_success "postgres.env template created"
log_todo "Edit ~/.config/claude/postgres.env with your PostgreSQL credentials (optional)"
fi
}
# --- Section 4: Validate Configuration ---
@@ -283,6 +308,7 @@ main() {
# Shared MCP servers at repository root
setup_shared_mcp "gitea"
setup_shared_mcp "netbox"
setup_shared_mcp "data-platform"
# Verify symlinks from plugins to shared MCP servers
verify_symlinks