chinese-llm-benchmark logo
chinese-llm-benchmark
★ 6.5k
vs
cap-evolve logo
cap-evolve
★ 66

chinese-llm-benchmark vs cap-evolve

chinese-llm-benchmark: ReLE Benchmark (formerly CLiB) provides a continuously updated evaluation for Chinese AI large language models, covering over 337 commercial and open-source LLMs. It offers multi-dimensional capability assessments across various domains, along with comprehensive rankings and a large defect library for model improvement.; cap-evolve: cap-evolve is an honesty-guaranteed framework that optimizes AI agent capabilities like prompts, tools, and skills by learning from failed evaluation traces. It automates the loop of evaluation, diagnosis, candidate generation, and validation, ensuring improvements are significant and honest.

01

TL;DR

chinese-llm-benchmark logo
chinese-llm-benchmarkLLM Infra · ★ 6.5k

Comparing and selecting the best performing LLMs for specific applications.

cap-evolve logo
cap-evolveLLM Infra · ★ 66

Optimizing AI agent capabilities when developing with Claude Code plugin.

02

Side-by-Side Comparison

Field
chinese-llm-benchmark logochinese-llm-benchmark
cap-evolve logocap-evolve
Category
LLM Infra
LLM Infra
Stars
★ 6.5k
★ 66
License
—
Apache-2.0
Install Command
git clone https://github.com/jeinlee1991/chinese-llm-benchmark.git
git clone https://github.com/skillberry-ai/cap-evolve.git && cd cap-evolve && python3 -m venv .venv && source .venv/bin/activate && pip install ./core
Updated
2d ago
1d ago
Open Source
Yes
Yes
Tags
LLM Evaluation, Chinese LLMs, AI Benchmark
AI Agent Optimization, Prompt Engineering, Tool Code Optimization
03

Features

chinese-llm-benchmark logochinese-llm-benchmark
01Extensive coverage of 337+ commercial and open-source Chinese LLMs.
02Multi-dimensional evaluation across 7 main domains and ~300 sub-dimensions.
03Provides detailed ranking lists for various capabilities and specific domains.
04Offers a large defect library with over 2 million LLM flaws for research and improvement.
05Supports customized model selection and free evaluation services for private models.
cap-evolve logocap-evolve
01Optimizes more than prompts, including executable tool code and entire skill packages.
02Learns from real agent failures and per-task causal feedback to make robust improvements.
03Ensures honest evaluation through a significance gate and a sealed, scored-once test split.
04Provides a dashboard to inspect every change, cost, timing, diff, and pass/fail heatmap.
04

Use Cases

chinese-llm-benchmark logochinese-llm-benchmark
↳Comparing and selecting the best performing LLMs for specific applications.
↳Identifying weaknesses and improving the capabilities of large language models.
↳Benchmarking private or custom LLMs against public models for performance and cost optimization.
cap-evolve logocap-evolve
↳Optimizing AI agent capabilities when developing with Claude Code plugin.
↳Integrating and refining agents from various coding-agent hosts like Codex, Gemini, or Droid.
↳Custom integration for developers who want to wire their own agent adapters and drive the CLI directly.
↳Applying pre-built adapter templates for popular benchmarks and data formats (e.g., JSONL, HuggingFace, tau2-bench).
05

Best For

chinese-llm-benchmark logochinese-llm-benchmark
EssentialTrending
cap-evolve logocap-evolve
—
FAQ

FAQ

What is the difference between chinese-llm-benchmark and cap-evolve?
Both chinese-llm-benchmark and cap-evolve are in the LLM Infra category. chinese-llm-benchmark has 6.5k stars, while cap-evolve has 66 stars.
Which is better, chinese-llm-benchmark or cap-evolve?
The best choice depends on your use case. Choose chinese-llm-benchmark if Comparing and selecting the best performing LLMs for specific applications., and cap-evolve if Optimizing AI agent capabilities when developing with Claude Code plugin..
Is chinese-llm-benchmark free or open source?
Yes, chinese-llm-benchmark is open source on GitHub.
Is cap-evolve free or open source?
Yes, cap-evolve is open source on GitHub (Apache-2.0).
Cloud Infrastructure
Ad

Deploy on DigitalOcean

Host your MCP servers, Docker containers, and AI agents with $200 free credit over 60 days.

Claim $200 Free Credit
→

Related