personal-projects/leo-claude-mktplace
Template
0
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implement validation tools (validate_compatibility, validate_agent_refs, validate_data_flow) #187

New Issue
Closed
opened 2026-01-26 19:02:11 +00:00 by lmiranda · 1 comment
lmiranda commented 2026-01-26 19:02:11 +00:00
Owner
Copy Link

Overview

Implement the core validation tools that check contracts and compatibility.

Tools

validate_compatibility

  • Input: Two plugin interfaces
  • Output: Compatibility report (what A produces that B accepts, gaps)
  • Compare outputs of plugin A with inputs of plugin B
  • Identify missing interfaces and type mismatches

validate_agent_refs

  • Input: Agent definition, list of available plugins
  • Output: Missing tools, invalid sequences
  • Verify all tool references exist in available plugins
  • Check tool naming conventions (server.tool format)

validate_data_flow

  • Input: Agent sequence
  • Output: Verification that each step's output matches next step's expected input
  • Trace data flow through agent tool chain
  • Identify type mismatches between steps

Issue Types to Detect

Type Severity Example
Missing tool Error Agent references pandas-mcp.transform but tool is pandas-mcp.reshape
Interface mismatch Error viz-platform expects chart_data but data-platform produces data_ref
Optional dependency Warning Agent uses dbt-mcp which may not be loaded
Undeclared output Warning Plugin produces output not listed in README

Sprint: Sprint 2 - contract-validator Plugin
Labels: Type/Feature, Complexity/High, Priority/High, Effort/L

## Overview Implement the core validation tools that check contracts and compatibility. ## Tools ### validate_compatibility - **Input:** Two plugin interfaces - **Output:** Compatibility report (what A produces that B accepts, gaps) - Compare outputs of plugin A with inputs of plugin B - Identify missing interfaces and type mismatches ### validate_agent_refs - **Input:** Agent definition, list of available plugins - **Output:** Missing tools, invalid sequences - Verify all tool references exist in available plugins - Check tool naming conventions (server.tool format) ### validate_data_flow - **Input:** Agent sequence - **Output:** Verification that each step's output matches next step's expected input - Trace data flow through agent tool chain - Identify type mismatches between steps ## Issue Types to Detect | Type | Severity | Example | |------|----------|---------| | Missing tool | Error | Agent references `pandas-mcp.transform` but tool is `pandas-mcp.reshape` | | Interface mismatch | Error | viz-platform expects `chart_data` but data-platform produces `data_ref` | | Optional dependency | Warning | Agent uses dbt-mcp which may not be loaded | | Undeclared output | Warning | Plugin produces output not listed in README | **Sprint:** Sprint 2 - contract-validator Plugin **Labels:** Type/Feature, Complexity/High, Priority/High, Effort/L
lmiranda added this to the Sprint 2 - contract-validator Plugin milestone 2026-01-26 19:02:40 +00:00
lmiranda commented 2026-01-26 19:50:27 +00:00
Author
Owner
Copy Link

Implementation Complete

Branch: feat/187-validation-tools
Commit: 46af000

Tools Implemented

validate_compatibility

  • Compares tool and command names between plugins
  • Identifies naming conflicts and shared tools
  • Finding: data-platform and projman share /initial-setup command!

validate_agent_refs

  • Checks agent tool references against available plugins
  • Reports missing tools and undocumented references

validate_data_flow

  • Validates data flow through agent tool sequences
  • Checks producer/consumer patterns (e.g., data_ref)

Issue Types Detected

Type Severity Description
missing_tool ERROR Tool reference doesn't exist
interface_mismatch ERROR/WARNING Command conflict or shared names
optional_dependency WARNING May not be loaded
undeclared_output WARNING/INFO Not documented

Test Results

validate_compatibility(data-platform, projman):
  Compatible: False (command conflict: /initial-setup)

validate_agent_refs(Planner):
  Valid: True (no documented tool refs)

validate_data_flow(Orchestrator):
  Valid: True, 4 flow steps
## Implementation Complete **Branch:** `feat/187-validation-tools` **Commit:** `46af000` ### Tools Implemented #### `validate_compatibility` - Compares tool and command names between plugins - Identifies naming conflicts and shared tools - **Finding:** data-platform and projman share `/initial-setup` command! #### `validate_agent_refs` - Checks agent tool references against available plugins - Reports missing tools and undocumented references #### `validate_data_flow` - Validates data flow through agent tool sequences - Checks producer/consumer patterns (e.g., data_ref) ### Issue Types Detected | Type | Severity | Description | |------|----------|-------------| | `missing_tool` | ERROR | Tool reference doesn't exist | | `interface_mismatch` | ERROR/WARNING | Command conflict or shared names | | `optional_dependency` | WARNING | May not be loaded | | `undeclared_output` | WARNING/INFO | Not documented | ### Test Results ``` validate_compatibility(data-platform, projman): Compatible: False (command conflict: /initial-setup) validate_agent_refs(Planner): Valid: True (no documented tool refs) validate_data_flow(Orchestrator): Valid: True, 4 flow steps ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Component/API
API endpoints, contracts, and integration
 Component/Auth
Authentication and authorization
 Component/Backend
Backend service code and business logic
 Component/Database
Database schemas, migrations, queries
 Component/Deploy
Deployment, infrastructure, DevOps
 Component/Docs
Documentation and guides
 Component/Frontend
User interface and client-side code
 Component/Infra
Infrastructure and system configuration
 Component/Testing
Test infrastructure and frameworks
 Tech/Docker
Docker containers and compose
 Tech/FastAPI
FastAPI backend framework
 Tech/JavaScript
JavaScript/Node.js code
 Tech/PostgreSQL
PostgreSQL database
 Tech/Python
Python language and libraries
 Tech/Redis
Redis cache and pub/sub
 Tech/Vue
Vue.js frontend framework
Complexity/Complex
High uncertainty or many dependencies
 Complexity/Medium
Some unknowns or dependencies
 Complexity/Simple
Straightforward, well-understood
 Effort/L
3-5 days
 Effort/M
1-2 days
 Effort/S
2-4 hours
 Effort/XL
> 1 week
 Effort/XS
< 2 hours
 Priority/Critical
Immediate attention required
 Priority/High
Important, needs attention soon
 Priority/Low
Nice to have, can wait
 Priority/Medium
Normal priority
 Type/Bug
Something isn't working
 Type/Chore
Maintenance tasks
 Type/Documentation
Documentation improvements
 Type/Feature
New feature or request
 Type/Refactor
Code refactoring without changing behavior
 Type/Test
Testing related
No Label
Milestone
No items
No Milestone Sprint 2 - contract-validator Plugin
Projects
Clear projects
No project
Assignees
Clear assignees
lmiranda
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: personal-projects/leo-claude-mktplace#187
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?