Best Open-Source AI Agent Observability and Tracing Tools
Once an AI agent calls tools in a loop, plain HTTP logs stop explaining why a run failed, stalled or burned through tokens. You need trace spans across prompts and tool calls, reproducible evaluations, and—when agents touch production—tamper-evident audit trails.
This list brings together reviewed open-source tools across that stack: full tracing and eval platforms, MCP server inspectors, local coding-agent cost dashboards, agent benchmarks and audit databases.
AgentIndex · Updated 2026-11-21
Pick by whether you need end-to-end tracing and evals, local coding-agent telemetry, or production audit and alerting.
- Local coding-agent telemetry
Transcripts and token spend in SQLite.
Quick picks
- lmnr — Apache-2.0 agent observability platform with end-to-end tracing, evaluations and SQL access over trace data.
- inspector — focused workbench for debugging, inspecting and evaluating MCP servers and MCP apps.
- AgentBench — Apache-2.0 containerised benchmark for comparing LLMs as agents across multi-turn environments.
- Claude-Code-Agent-Monitor — local-first dashboard tracking Claude Code, Cursor and Codex transcripts, token cost estimates and session resumption.
- ZizkaDB — tamper-evident audit trail database with session replay built for agent accountability and compliance.
- langfuse-mcp — MIT-licensed MCP server exposing 37 Langfuse tools so agents can inspect traces, scores and prompts directly.
Side-by-side comparison
| Tool | What it is | Stars | License | Last push | Maintenance | Releases, 90 days | Commits, 12 weeks | Install |
|---|---|---|---|---|---|---|---|---|
| lmnr | Open-source observability platform for AI agents: tracing, evals and SQL access | 3.5k | Apache-2.0 | 2026-11-20 | Active | 8 | git clone https://github.com/lmnr-ai/lmnr | |
| inspector | Development platform for debugging, testing and evaluating MCP servers and apps | 2.3k | n/a | 2026-11-21 | Active | 30 | npx @mcpjam/inspector@latest | |
| AgentBench | Tsinghua's Apache-2.0 benchmark for evaluating LLMs as agents, now in a function-calling version with containerised tasks | 3.8k | Apache-2.0 | 2026-02-08 | Slowing | 0 | pip install -r requirements.txt | |
| Claude-Code-Agent-Monitor | Local-first observability dashboard and session tracker for Claude Code, Cursor, and Codex with transcript visualization, token cost estimation, and task resumption | 1.2k | MIT | 2026-11-21 | Active | 22 | git clone https://github.com/hoangsonww/Claude-Code-Agent-Monitor.git | |
| codex-usage-tracker | Local-first observability dashboard and SQLite analytics tracker for OpenAI Codex token consumption and cost profiling. | 196 | MIT | 2026-10-14 | Active | 0 | python -m pip install --user pipx | |
| ZizkaDB | Audit trail database for AI agents with tamper-evident logs, session replay, and EU AI Act Article 12 compliance. | 163 | n/a | 2026-11-21 | Active | 9 | curl -fsSL https://raw.githubusercontent.com/Zizka-ai/ZizkaDB/main/scripts/qu… | |
| langfuse-mcp | Production-grade Model Context Protocol server for Langfuse observability with 37 tools across traces, sessions, scores, and prompts | 112 | MIT | 2026-11-06 | Active | 6 | pip install langfuse-mcp | |
| keep | Open-source alert management and AIOps platform with YAML workflows | 12.5k | n/a | 2026-11-20 | Active | 2 | git clone https://github.com/keephq/keep.git |
Tools in this list with an in-depth AgentIndex review: 8/8
How to choose
- Separate production tracing from developer debugging: Laminar covers live traces and SQL queries over runs, while MCPJam Inspector focuses on testing MCP servers during development.
- If you mainly track local coding agents (Claude Code, Cursor or Codex), a local-first SQLite monitor avoids sending transcripts to an external backend.
- For compliance or high-stakes workflows, look for immutable audit logs and session replay rather than metrics alone.
- Check hosting overhead and licences: Apache-2.0 and MIT dominate here, while a few projects carry custom or unspecified licences.
FAQ
Which open-source tool offers end-to-end AI agent tracing for free?
Laminar (lmnr) is Apache-2.0 and built specifically for AI agent tracing, evaluations and SQL queries over trace records, so you can self-host it without per-seat fees. If you already run Langfuse, langfuse-mcp lets your coding agent query those traces over MCP.
How do I monitor token spend for Claude Code or Codex locally?
Claude-Code-Agent-Monitor and codex-usage-tracker both run locally on SQLite to visualise transcripts, sessions and estimated token spend without routing your code to a third-party cloud.
When is an audit database better than standard tracing?
Standard tracing helps engineers debug latency and prompts; an audit database like ZizkaDB adds tamper-evident checksums and session replay when you need verifiable decision records for compliance or incident review.
How we compiled this list
Tools are chosen by hand from the projects in our index, favouring those we have reviewed in depth. Stars, licence, last-push date and install command come from each repository and are refreshed with the site data. The maintenance column comes from the GitHub API (archived flag, last commit, releases in the last 90 days) and is refreshed daily. We read each project's README and docs; we have not run every tool ourselves, so test before you depend on one.
More lists
- Open-Source Multi-Agent Frameworks: CrewAI, Pydantic AI, Google ADK, Microsoft Agent Framework & More
- Self-Hosted AI Workflow Automation & Durable Execution: n8n, Activepieces, Temporal, Conductor & More
- Open-Source RAG Platforms & Deep Research Agents: RAGFlow, MaxKB, GPT Researcher, DeerFlow & More
- SDKs, Frameworks & Official Reference Servers for Building with MCP
- Best MCP Servers for Databases
- Best MCP Servers for Browser Automation