MinHash Text Deduplicator

MinHash Text Deduplicator MCP Connector for Claude

A+

Detect near-duplicate texts using MinHash signatures and Jaccard similarity.

3 tools Official Updated Oct 1, 2026 Official Vinkius Partner

This MCP server provides tools to identify redundant content within large text collections. By using shingling and deterministic MinHash signatures, it estimates Jaccard similarity to find near-duplicates. Use compute_similarity_matrix to see how all texts relate, identify_duplicate_clusters to group similar items, or check_is_duplicate to verify if a specific text already exists in your library.

minhashjaccard-similaritydeduplicationtext-analysisshingling

3 tools expose this connector's capabilities to your AI agent.

check_is_duplicate

Performs a binary check to determine if a new piece of text is a near-duplicate of any existing text

compute_similarity_matrix

Calculates the estimated Jaccard similarity between all provided text entries

identify_duplicate_clusters

Groups texts into distinct sets of near-duplicates based on a specific similarity requirement

See how to talk to your AI agent using MinHash Text Deduplicator.

Find all groups of duplicate texts in this list: ['Hello world', 'Hello world!', 'Goodbye moon', 'Goodbye moon!'] with a threshold of 0.8.

[[0, 1], [2, 3]]

Is 'The quick brown fox' a duplicate of ['The quick brown fox jumps over the lazy dog'] with a threshold of 0.5?

No, the similarity is below the threshold.

Show me the similarity matrix for these three sentences: ['A', 'B', 'C'].

[[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]

It uses MinHash signatures to estimate the Jaccard similarity between sets of shingles, providing a score between 0.0 and 1.0.

Related Connectors