Best Open-Source AI Agent Observability and Tracing Tools

Once an AI agent calls tools in a loop, plain HTTP logs stop explaining why a run failed, stalled or burned through tokens. You need trace spans across prompts and tool calls, reproducible evaluations, and—when agents touch production—tamper-evident audit trails.

This list brings together reviewed open-source tools across that stack: full tracing and eval platforms, MCP server inspectors, local coding-agent cost dashboards, agent benchmarks and audit databases.

AgentIndex · Updated 2026-11-21

Agent observability tools by job

Pick by whether you need end-to-end tracing and evals, local coding-agent telemetry, or production audit and alerting.

  1. Tracing & evaluations

    Traces, MCP testing and agent benchmarks.

  2. Local coding-agent telemetry

    Transcripts and token spend in SQLite.

  3. Audit trails & AIOps alerts

    Tamper-evident logs and incident workflows.

Quick picks

  • lmnr — Apache-2.0 agent observability platform with end-to-end tracing, evaluations and SQL access over trace data.
  • inspector — focused workbench for debugging, inspecting and evaluating MCP servers and MCP apps.
  • AgentBench — Apache-2.0 containerised benchmark for comparing LLMs as agents across multi-turn environments.
  • Claude-Code-Agent-Monitor — local-first dashboard tracking Claude Code, Cursor and Codex transcripts, token cost estimates and session resumption.
  • ZizkaDB — tamper-evident audit trail database with session replay built for agent accountability and compliance.
  • langfuse-mcp — MIT-licensed MCP server exposing 37 Langfuse tools so agents can inspect traces, scores and prompts directly.

Side-by-side comparison

ToolWhat it isStarsLicenseLast pushMaintenanceReleases, 90 daysCommits, 12 weeksInstall
lmnrOpen-source observability platform for AI agents: tracing, evals and SQL access3.5kApache-2.02026-11-20Active8Commits, 12 weeks: 21, 13, 14, 39, 21, 26, 20, 23, 28, 32, 22, 0git clone https://github.com/lmnr-ai/lmnr
inspectorDevelopment platform for debugging, testing and evaluating MCP servers and apps2.3kn/a2026-11-21Active30Commits, 12 weeks: 174, 157, 172, 219, 257, 225, 145, 221, 240, 75, 133, 199npx @mcpjam/inspector@latest
AgentBenchTsinghua's Apache-2.0 benchmark for evaluating LLMs as agents, now in a function-calling version with containerised tasks3.8kApache-2.02026-02-08Slowing0Commits, 12 weeks: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0pip install -r requirements.txt
Claude-Code-Agent-MonitorLocal-first observability dashboard and session tracker for Claude Code, Cursor, and Codex with transcript visualization, token cost estimation, and task resumption1.2kMIT2026-11-21Active22Commits, 12 weeks: 11, 32, 20, 28, 15, 54, 15, 60, 17, 19, 22, 1git clone https://github.com/hoangsonww/Claude-Code-Agent-Monitor.git
codex-usage-trackerLocal-first observability dashboard and SQLite analytics tracker for OpenAI Codex token consumption and cost profiling.196MIT2026-10-14Active0Commits, 12 weeks: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0python -m pip install --user pipx
ZizkaDBAudit trail database for AI agents with tamper-evident logs, session replay, and EU AI Act Article 12 compliance.163n/a2026-11-21Active9Commits, 12 weeks: 38, 20, 7, 26, 8, 8, 11, 51, 84, 77, 44, 29curl -fsSL https://raw.githubusercontent.com/Zizka-ai/ZizkaDB/main/scripts/qu…
langfuse-mcpProduction-grade Model Context Protocol server for Langfuse observability with 37 tools across traces, sessions, scores, and prompts112MIT2026-11-06Active6Commits, 12 weeks: 0, 7, 0, 9, 0, 1, 2, 1, 2, 1, 0, 0pip install langfuse-mcp
keepOpen-source alert management and AIOps platform with YAML workflows12.5kn/a2026-11-20Active2Commits, 12 weeks: 2, 1, 7, 2, 0, 1, 0, 0, 1, 0, 7, 3git clone https://github.com/keephq/keep.git

Tools in this list with an in-depth AgentIndex review: 8/8

How to choose

  • Separate production tracing from developer debugging: Laminar covers live traces and SQL queries over runs, while MCPJam Inspector focuses on testing MCP servers during development.
  • If you mainly track local coding agents (Claude Code, Cursor or Codex), a local-first SQLite monitor avoids sending transcripts to an external backend.
  • For compliance or high-stakes workflows, look for immutable audit logs and session replay rather than metrics alone.
  • Check hosting overhead and licences: Apache-2.0 and MIT dominate here, while a few projects carry custom or unspecified licences.

FAQ

Which open-source tool offers end-to-end AI agent tracing for free?

Laminar (lmnr) is Apache-2.0 and built specifically for AI agent tracing, evaluations and SQL queries over trace records, so you can self-host it without per-seat fees. If you already run Langfuse, langfuse-mcp lets your coding agent query those traces over MCP.

How do I monitor token spend for Claude Code or Codex locally?

Claude-Code-Agent-Monitor and codex-usage-tracker both run locally on SQLite to visualise transcripts, sessions and estimated token spend without routing your code to a third-party cloud.

When is an audit database better than standard tracing?

Standard tracing helps engineers debug latency and prompts; an audit database like ZizkaDB adds tamper-evident checksums and session replay when you need verifiable decision records for compliance or incident review.

How we compiled this list

Tools are chosen by hand from the projects in our index, favouring those we have reviewed in depth. Stars, licence, last-push date and install command come from each repository and are refreshed with the site data. The maintenance column comes from the GitHub API (archived flag, last commit, releases in the last 90 days) and is refreshed daily. We read each project's README and docs; we have not run every tool ourselves, so test before you depend on one.

More lists