feat: add data-platform plugin (v4.0.0)
Add new data-platform plugin for data engineering workflows with: MCP Server (32 tools): - pandas operations (14 tools): read_csv, read_parquet, read_json, to_csv, to_parquet, describe, head, tail, filter, select, groupby, join, list_data, drop_data - PostgreSQL/PostGIS (10 tools): pg_connect, pg_query, pg_execute, pg_tables, pg_columns, pg_schemas, st_tables, st_geometry_type, st_srid, st_extent - dbt integration (8 tools): dbt_parse, dbt_run, dbt_test, dbt_build, dbt_compile, dbt_ls, dbt_docs_generate, dbt_lineage Plugin Features: - Arrow IPC data_ref system for DataFrame persistence across tool calls - Pre-execution validation for dbt with `dbt parse` - SessionStart hook for PostgreSQL connectivity check (non-blocking) - Hybrid configuration (system ~/.config/claude/postgres.env + project .env) - Memory management with 100k row limit and chunking support Commands: /initial-setup, /ingest, /profile, /schema, /explain, /lineage, /run Agents: data-ingestion, data-analysis Test suite: 71 tests covering config, data store, pandas, postgres, dbt tools Addresses data workflow issues from personal-portfolio project: - Lost data after multiple interactions (solved by Arrow IPC data_ref) - dbt 1.9+ syntax deprecation (solved by pre-execution validation) - Ungraceful PostgreSQL error handling (solved by SessionStart hook) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
44
plugins/data-platform/commands/profile.md
Normal file
44
plugins/data-platform/commands/profile.md
Normal file
@@ -0,0 +1,44 @@
|
||||
# /profile - Data Profiling
|
||||
|
||||
Generate statistical profile and quality report for a DataFrame.
|
||||
|
||||
## Usage
|
||||
|
||||
```
|
||||
/profile <data_ref>
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
1. **Get data reference**:
|
||||
- If no data_ref provided, use `list_data` to show available options
|
||||
- Validate the data_ref exists
|
||||
|
||||
2. **Generate profile**:
|
||||
- Use `describe` for statistical summary
|
||||
- Analyze null counts, unique values, data types
|
||||
|
||||
3. **Quality assessment**:
|
||||
- Identify columns with high null percentage
|
||||
- Flag potential data quality issues
|
||||
- Suggest cleaning operations if needed
|
||||
|
||||
4. **Report**:
|
||||
- Summary statistics per column
|
||||
- Data type distribution
|
||||
- Memory usage
|
||||
- Quality score
|
||||
|
||||
## Examples
|
||||
|
||||
```
|
||||
/profile sales_data
|
||||
/profile df_a1b2c3d4
|
||||
```
|
||||
|
||||
## Available Tools
|
||||
|
||||
Use these MCP tools:
|
||||
- `describe` - Get statistical summary
|
||||
- `head` - Preview first rows
|
||||
- `list_data` - List available DataFrames
|
||||
Reference in New Issue
Block a user