Output Format Stability Checker

Output Format Stability Checker MCP Connector for Claude

A+

Detect non-deterministic shifts in LLM response structures.

3 tools Official Updated Oct 1, 2026 Official Vinkius Partner

This MCP server provides tools to measure the structural consistency of Large Language Model (LLM) responses. It helps developers detect non-deterministic shifts in formatting--such as changes in JSON key order, Markdown header hierarchy, or code block language usage--even when the semantic content remains valid. Use analyze_output_stability to evaluate a batch of responses, extract_structural_features to decompose a single response into its markers, or get_structural_summary to identify dominant patterns across multiple runs.

llmstabilityjsonmarkdowntestingdeterminism

3 tools expose this connector's capabilities to your AI agent.

extract_structural_features

Decomposes a raw text string into a set of discrete structural markers

analyze_output_stability

Evaluates a collection of LLM responses to determine if their structural formats are consistent

get_structural_summary

Provides a high-level overview of the most common structural patterns found in a set of responses

See how to talk to your AI agent using Output Format Stability Checker.

Check if these three JSON responses are structurally stable with a threshold of 0.8: [{"a":1, "b":2}, {"b":2, "a":1}, {"a":1, "b":2}]

{"isStable": true, "unstableResponseIndices": [], "similarityMatrix": [[1.0, 1.0, 1.0], [1.0, 1.0, 1.0], [1.0, 1.0, 1.0]]}

Extract the structural features from this markdown text: '# Title\n\n```python\nprint(1)\n```'

{"jsonKeys": [], "headerHierarchy": ["h1"], "codeBlockLanguages": ["python"]}

What are the dominant JSON keys in these responses: ["{\"id\":1}", "{\"id\":2}", "{\"name\":3}"]

{"dominantJsonKeys": ["id"], "commonHeaderPatterns": [], "dominantLanguages": []}

It refers to whether the 'shape' of an LLM response (like JSON keys or Markdown headers) remains consistent across different generations.

Related Connectors