AgentIndex icon
AgentIndex
coder_eval logo
coder_eval
★ 154
vs
vurb.ts logo
vurb.ts
★ 255

coder_eval vs vurb.ts

coder_eval: Coder Eval runs a real coding agent (Claude Code, Codex, Antigravity, OpenCode or Pi) in a sandbox against declarative YAML tasks and scores the files and commands it produced. It supports weighted 0.0-1.0 criteria, skill-trigger checks, A/B experiments and cost telemetry, and can gate CI on agent quality.; vurb.ts: Vurb.ts is a TypeScript framework for building production-grade MCP servers using the MVA (Model-View-Agent) pattern. It provides structured perception, data security, and anti-hallucination mechanisms. Developers can quickly scaffold servers with file-based routing, Presenters, and built-in features like PII redaction and FSM state gates.

01

TL;DR

coder_eval logo
coder_evalDev Tooling · ★ 154

Benchmark coding agents on your own domain tasks

Read coder_eval full review →
vurb.ts logo
vurb.tsDev Tooling · ★ 255

Build production-grade MCP servers from scratch

Read vurb.ts full review →
02

Side-by-Side Comparison

Field
coder_eval logocoder_eval
vurb.ts logovurb.ts
Category
Dev Tooling
Dev Tooling
Stars
★ 154
★ 255
License
Apache-2.0
Apache-2.0
Install Command
uv tool install coder-eval
npm install -g @vurb/core
Updated
2d ago
1mo ago
Open Source
Yes
Yes
Website
↗ Visit
↗ Visit
GitHub
↗ GitHub
↗ GitHub
Tags
agent-evaluation, agent-skills, agent-testing
ai-tools, anthropic, claude-code
03

Features

coder_eval logocoder_eval
01Declarative YAML tasks with pinned dependencies and success criteria
02Sandboxed execution with resource limits
03Weighted 0.0-1.0 scoring, including LLM-graded rubrics and skill_triggered checks
04A/B experiments across agents, models, tools and prompts
05Telemetry for every tool call, token count and cost
vurb.ts logovurb.ts
01File-based routing: auto-discover tools from file structure
02Presenter egress firewall: schema-based whitelist security
03PII redaction with Late Guillotine pattern
04FSM State Gate for temporal anti-hallucination
05Zero-trust sandboxed computation delegation
04

Use Cases

coder_eval logocoder_eval
↳Benchmark coding agents on your own domain tasks
↳Verify that a skill actually triggers in the agent you ship for
↳Fail CI builds when agent or skill quality regresses
vurb.ts logovurb.ts
↳Build production-grade MCP servers from scratch
↳Bridge existing REST APIs to MCP using OpenAPI generator
↳Scaffold database-driven MCP servers with Prisma integration
FAQ

FAQ

What is the difference between coder_eval and vurb.ts?
Both coder_eval and vurb.ts are in the Dev Tooling category. coder_eval has 154 stars, while vurb.ts has 255 stars.
Which is better, coder_eval or vurb.ts?
The best choice depends on your use case. Choose coder_eval if Benchmark coding agents on your own domain tasks, and vurb.ts if Build production-grade MCP servers from scratch.
Is coder_eval free or open source?
Yes, coder_eval is open source on GitHub (Apache-2.0).
Is vurb.ts free or open source?
Yes, vurb.ts is open source on GitHub (Apache-2.0).
Cloud Infrastructure
Ad

Deploy on DigitalOcean

Host your MCP servers, Docker containers, and AI agents with $200 free credit over 60 days.

Claim $200 Free Credit
→

Related

Alternatives to coder_eval →Alternatives to vurb.ts →coder_eval details →vurb.ts details →