1

lessons/sprints/sprint-2---contract-validator-plugin-implementation

Leo Miranda edited this page 2026-01-26 20:11:37 +00:00

Table of Contents

• Sprint 2 - contract-validator Plugin Implementation
• Metadata
• Context
• What Went Well
• 1. Real Issue Discovery
• 2. Comprehensive Test Coverage
• 3. Regex Pattern Refinement
• Challenges & Solutions
• Challenge 1: Plugin Name Matching in Reports
• Challenge 2: CLAUDE.md Agent Parsing Noise
• Challenge 3: Branch Protection Workflow
• Key Decisions
• Metrics
• Recommendations for Future Sprints
• Tags

Sprint 2 - contract-validator Plugin Implementation

Metadata

• Sprint: Sprint 2 - contract-validator Plugin
• Date: 2026-01-26
• Issues: #185-#193 (9 issues)
• Status: Completed

Context

Implemented the contract-validator plugin for cross-plugin compatibility validation and CLAUDE.md agent verification. This plugin addresses growing compatibility concerns as the marketplace expands to 12+ plugins.

What Went Well

1. Real Issue Discovery

The validation tools immediately found a real compatibility issue: /initial-setup command is defined by three plugins (projman, data-platform, pr-review). This validates the plugin's usefulness.

2. Comprehensive Test Coverage

Achieved 88% average coverage across core modules:

• parse_tools.py: 79%
• validation_tools.py: 96%
• report_tools.py: 89%

3. Regex Pattern Refinement

Iteratively improved parsing patterns through real-world testing against existing plugins. Key refinements:

• Agent table parsing required strict **AgentName** format to avoid noise
• Command parsing needed to exclude table separators (|---|)
• Section extraction needed to capture full content until next same-level header

Challenges & Solutions

Challenge 1: Plugin Name Matching in Reports

Problem: Report tools couldn't match parsed plugin names (e.g., "Plugin One") to directory names (e.g., "plugin-one") for pairwise validation.

Solution: Added _find_plugin_dir() method that checks:

1. Exact lowercase match of directory name
2. Plugin.json name field match

Prevention: When designing lookup functions, consider all naming variations upfront.

Challenge 2: CLAUDE.md Agent Parsing Noise

Problem: Initial regex picked up 62 "agents" including unrelated table rows.

Solution: Made pattern more strict:

• Require **BoldName** format in first column
• Check personality column doesn't contain backticks (tool names)
• Skip rows where personality contains "---" or looks like header

Prevention: Test parsers against real-world documents early, not just synthetic test data.

Challenge 3: Branch Protection Workflow

Problem: Development branch has push protection, breaking the merge workflow used in Sprint 1.

Solution: Each issue gets its own feature branch pushed to remote. PRs created via Gitea web interface.

Prevention: Check branch protection settings before starting sprint execution.

Key Decisions

1. 7 MCP Tools - Split into parse (2), validate (3), report (2) for clear separation
2. Pydantic Models - Used for all structured data (PluginInterface, ValidationIssue, etc.)
3. Dual Report Format - Markdown for human review, JSON for programmatic use
4. Known Data Flow Patterns - Hardcoded producer/consumer tool lists for data flow validation

Metrics

Metric	Value
Issues Completed	9/9
MCP Tools	7
Commands	3
Agents	2
Tests	34
Coverage	88% avg
Real Issues Found	1 (command conflict)

Recommendations for Future Sprints

1. Address Command Conflict - Create issue to rename /initial-setup in data-platform and pr-review to avoid conflict with projman
2. Add CI Integration - Run /validate-contracts in CI pipeline before merging plugin changes
3. Expand Data Flow Patterns - Add more producer/consumer tool mappings as plugins grow

Tags

• contract-validator
• mcp-server
• validation
• parsing
• compatibility

---

Tags: contract-validator, mcp-server, validation, parsing, sprint-2, v5.0.0