MCP For Claude Marketplace
LangSmith (LLM Observability & Hub)

LangSmith (LLM Observability & Hub) MCP Connector for Claude

A+

Monitor LLM apps via LangSmith — track traces, audit prompt templates, and manage evaluation datasets.

6 tools Official Updated Oct 1, 2026 Official Vinkius Partner
More Details Connect to Claude

Connect your LangSmith account to any AI agent and take full control of your LLM observability, tracing, and prompt management through natural conversation.

What you can do

  • Trace Orchestration — List active tracing projects and retrieve detailed execution logs for specific LLM invocation runs directly from your agent
  • Performance Telemetry — Extract precise metrics including token consumption, prompt latency, and exact error strings from your AI pipelines
  • Prompt Hub Access — Navigate and retrieve managed prompt templates, variable definitions, and version histories hosted in the LangChain Hub
  • Evaluation Datasets — Enumerate curated 'golden' datasets used for automated evaluation of prompt logic or few-shot injection models
  • Human-in-the-Loop Audit — Monitor active annotation queues where human reviewers assess the alignment, accuracy, and safety of generated LLM traces
  • Agentic Step Analysis — Deep-dive into multi-turn agentic workflows to understand nested tool calls and internal reasoning paths securely

How it works

  1. Subscribe to this server
  2. Enter your LangSmith API Key and Endpoint
  3. Start monitoring your LLM infrastructure from Claude, Cursor, or any MCP-compatible client

Who is this for?

  • LLM Engineers — debug complex agentic traces and measure prompt performance through natural conversation without manual UI filtering
  • AI Developers — retrieve the latest prompt templates from the Hub and verify evaluation dataset structures directly from your workspace
  • AI Analysts — audit human feedback queues and report on overall model grounding and accuracy across multiple tracing projects
llm-observabilitytracingprompt-managementevaluationai-debuggingllm-ops

6 tools expose this connector's capabilities to your AI agent.

list_annotation_queues

List active human-in-the-loop annotation queues

list_datasets

List all evaluation and fine-tuning datasets mapped in LangSmith

list_prompts

Extract prompt templates hosted in the LangChain Hub

list_runs

Isolates the raw interactions containing prompts sent to and responses received from the AI models. List explicit LLM invocation runs within a specific project

get_run

Get precise telemetry for a single LLM invocation run

list_projects

Maps out the boundaries of distinct AI pipelines currently monitored by LangSmith. List all active LangSmith tracing projects/sessions

See how to talk to your AI agent using LangSmith (LLM Observability & Hub).

List all active tracing projects in LangSmith

I've found 4 active tracing projects: 'Production-Bot-V2', 'Staging-RAG-Explorer', 'Customer-Support-Agent', and 'Internal-Testing-Suite'. Which one would you like to see the recent runs for?

Show me the telemetry for the last run in the 'Production-Bot-V2' project

Retrieving data for run ID 'run-98765'… The execution was successful. Total tokens: 1,420 (Prompt: 850, Completion: 570). Latency: 2.4s. The agent successfully used the 'Web-Search' tool before generating the answer. No errors detected.

List all prompts hosted in our Hub repository

I've identified 3 prompts in your Hub: 'summarization-agent-v1', 'customer-service-v3', and 'data-extraction-helper'. I can retrieve the full instruction text and version history for any of these.

Yes. Use the `get_run_telemetry` tool with a specific Run ID. Your agent will retrieve the exact token count (prompt + completion) and latency metrics calculated by LangSmith for that interaction.

Related Connectors

Humanloop (LLM Prompt Management API)

Humanloop (LLM Prompt Management API) MCP

12 tools Official

Manage, version, and deploy LLM prompts directly from your AI agent using the Humanloop API.

A+ View details →
Replicate

Replicate MCP

12 tools

Equip your AI to dynamically search, run, and monitor thousands of open-source machine learning models hosted on Replicate via simple text commands.

A+ View details →
LangSmith

LangSmith MCP

3 tools Official

Observability and evaluation platform for LLM applications — monitor traces, debug agent runs, and track performance metrics across your AI stack.

A+ View details →
Cerebras Inference

Cerebras Inference MCP

15 tools Official

Access lightning-fast AI inference via Cerebras Wafer-Scale Engine — generate chat completions, manage models, and run batch jobs at record speeds.

A+ View details →