shaxzodbek-uzb/mcp-vitals

mcp-vitals

Vital signs for your MCP server. Inspect capabilities, benchmark tool-call latency (p50/p95/p99), and assert health in CI — the ab / k6 / pytest for Model Context Protocol servers. Non-interactive, scriptable, no LLM key required.

<p align="center" <img src="https://raw.githubusercontent.com/shaxzodbek-uzb/mcp-vitals/main/assets/hero.png" alt="mcp-vitals — benchmark MCP tool-call latency (p50/p95/p99) and gate CI on a latency budget" width="900" </p

# Benchmark a tool's latency and fail CI if it's too slow — no install, no API key
npx mcp-vitals bench --tool search --fail-on 'p95<200ms' npx your-mcp-server

---

Why mcp-vitals?

There are great tools for exploring an MCP server interactively. There is nothing for measuring one and gating CI on the result.

| | Inspector | mcpjam | mcp-probe | mcp-vitals | |---|:---:|:---:|:---:|:---:| | Inspect tools / resources / prompts | ✅ | ✅ | ✅ | ✅ | | One-shot tool call | ✅ | ✅ | ✅ | ✅ | | Latency percentiles (p50/p95/p99) | ❌ | ❌ | ❌ | ✅ | | Concurrency / throughput load | ❌ | ❌ | ❌ | ✅ | | Gate a PR on a latency budget | ❌ | ❌ | ❌ | ✅ | | Gate a PR on a latency regression | ❌ | ❌ | ❌ | ✅ | | LLM-free (no API key) | ✅ | ❌ | ✅ | ✅ | | npx-installable | ✅ | ✅ | cargo | ✅ | | JUnit + JSON reporters | ❌ | ✅ | partial | ✅ |

mcp-vitals measures the real tools/call round-trip with process.hrtime — pure protocol latency, no model inference mixed in — and lets you commit a mcp-vitals.yaml that fails the build when a tool goes missing, ships an invalid schema, or blows its latency budget.

Install

npx mcp-vitals --help        # zero-install
npm i -g mcp-vitals          # or install globally

Requires Node.js ≥ 20.

Commands

mcp-vitals connects to any MCP server over stdio (a launch command), Streamable HTTP, or SSE (--url).

Put mcp-vitals options before the server command: mcp-vitals bench --tool x -n 100 npx your-server.

bench — latency benchmark (the headline)

# 200 iterations, 5 warmup, fail the command if p95 exceeds 200ms
mcp-vitals bench --tool search --args '{"q":"test"}' \
  -n 200 -w 5 --fail-on 'p95<200ms' --fail-on 'errorRate<=0' \
  npx your-mcp-server

# Load test: keep 10 calls in flight
mcp-vitals bench --tool search -c 10 -n 500 npx your-mcp-server

# Baseline raw transport overhead with a no-arg probe
mcp-vitals bench --probe listTools npx your-mcp-server

Reports min / mean / p50 / p90 / p95 / p99 / max / stddev, cold-start, throughput, and error rate. --json emits the full distribution for dashboards.