MCP For Claude Marketplace
A/B Test Significance Calculator

A/B Test Significance Calculator MCP Connector for Claude

A+

Calculate statistical significance, p-values, and uplift for A/B tests.

3 tools Official Updated Jun 29, 2026 Official Vinkius Partner
More Details Connect to Claude

This MCP server provides a specialized engine for evaluating A/B test results. By connecting your AI agent to this tool, you can instantly compute critical experimental metrics including p-values, confidence intervals (90%, 95%, and 99%), relative and absolute uplift, and statistical power. Use calculate_significance_metrics to determine the probability of observed differences occurring by chance, calculate_uplift_analysis to measure the magnitude of change between groups, and generate_experiment_verdict to receive actionable business recommendations on whether to end a test or continue collecting data.

ab-testingstatisticsp-valueconversion-rateexperimentation

3 tools expose this connector's capabilities to your AI agent.

generate_experiment_verdict

Provides a business recommendation on whether to end the test or continue collecting data

calculate_significance_metrics

Calculates core probability metrics and confidence ranges for the experiment

calculate_uplift_analysis

Computes the magnitude of change between the control and variant groups

See how to talk to your AI agent using A/B Test Significance Calculator.

I have an A/B test where the control group had 1000 visitors and 50 conversions, while the variant group had 1100 visitors and 70 conversions. What is the significance?

The p-value for this test is approximately 0.145, which suggests that the observed difference is not statistically significant at the 95% confidence level.

Calculate the uplift for a control group with 5000 visitors and 200 conversions, and a variant group with 5200 visitors and 250 conversions.

The relative uplift is approximately 12.5%, and the absolute uplift is an increase of 0.96 percentage points.

Based on a p-value of 0.03 and a statistical power of 0.85, should I end my experiment with a significance threshold of 0.05?

Yes, you should end the test. The result is conclusive because the p-value is below the threshold and the power meets the required level.

You can use the `calculate_significance_metrics` tool. It will return a p-value; if this value is below your significance threshold (typically 0.05), the result is considered statistically significant.

Related Connectors

AE

AI Ethics Prover MCP

1 tools Official

An AI said 'AI should be fair and transparent' without naming a single affected group. It said 'we checked for bias' without naming a metric, attribute, or measured disparity. It said 'contact support' as the recourse mechanism. That is not ethics — that is ethics washing. This tool forces five operational axes: stakeholder identification, harm quantification, bias auditing, transparency demonstration, and recourse mechanisms.

A+ View details →
NEW
CC

Canonical Checker MCP

3 tools Official

Identify the authoritative URL version and detect SEO duplicate content risks.

A+ View details →
NEW
IS

Image SEO Auditor MCP

3 tools Official

Automated analysis of image metadata to identify SEO and accessibility violations.

A+ View details →
NEW
PW

Prevailing Wage Calculator MCP

3 tools Official

Lookup Davis-Bacon Act compliant wage rates and fringe benefits for construction trades.

A+ View details →