AgentIndex icon
AgentIndex
ToolsCategoriesTrendingNewCompare
Submit Tool
ToolsCategoriesTrendingNewCompare
Home/
Browse AI Tool Categories/
LLM Infra

LLM Infra (2026)

Infrastructure tools for deploying, serving, fine-tuning, and evaluating large language models at scale.

68 tools

LLM Infra brings together the most relevant AI tools, MCP servers, and developer products for this topic on AgentIndex. It is designed to help you scan the landscape quickly before narrowing options by stars, licenses, freshness, and practical fit.

This page currently includes 68 tools in the category, making it useful for shortlisting open-source options, comparing popular projects, and building an initial evaluation set before deeper product testing.

NameTaglineCategoryStarsLicenseUpdated
© 2026 AgentIndex.app|Built by a 10-year iOS Developer.
QYSGitHubBuy me a coffee ☕

Browse by Category

Code AssistantWorkflow AutomationRAG / Knowledge BaseMulti-AgentBrowser AutomationLLM InfraDev ToolingObservability

Not affiliated with Anthropic, OpenAI or Microsoft.

MetaGPT logo
MetaGPTMost PopularTrending···🌟 The Multi-Agent Framework: First AI Software Company, Towards Natural Language ProgrammingLLM Infra★ 70.3kMIT2026-01-21
cua logo
cuaMost PopularTrending···Open-source infrastructure for Computer-Use Agents. Sandboxes, SDKs, and benchmarks to train and evaluate AI agents that can control full desktops (macOS, Linux, Windows).LLM Infra★ 22.5kMIT2026-09-10
xLAM logo
xLAMTrendingxLAM: A Family of Large Action Models to Empower AI Agent SystemsLLM Infra★ 638APACHE2026-06-02
on-policy logo
on-policyTrendingReinforcement Learning···This is the official implementation of Multi-Agent PPO (MAPPO).LLM Infra★ 2.1kMIT2024-07-18
youtu-agent logo
youtu-agentTrendingA simple yet powerful agent framework that delivers with open-source modelsLLM Infra★ 4.6kNOASSERTION2026-03-21
mini-swe-agent logo
mini-swe-agentTrendingHidden GemThe 100 line AI agent that solves GitHub issues or helps you in your command line. Radically simple, no huge configs, no giant monorepo—but scores >74% on SWE-bench verified!LLM Infra★ 7.3k—2026-09-07
FedML logo
FedMLTrendingEssentialFEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.LLM Infra★ 4.1kApache-2.02025-10-28
chatarena logo
chatarenaTrendingChatArena (or Chat Arena) is a Multi-Agent Language Game Environments for LLMs. The goal is to develop communication and collaboration capabilities of AIs.LLM Infra★ 1.6kAPACHE2025-08-11
AgileRL logo
AgileRLTrendingHidden GemStreamlining reinforcement learning with RLOps. State-of-the-art RL algorithms and tools, with 10x faster training through evolutionary hyperparameter optimization.LLM Infra★ 951—2026-09-10
AReaL logo
AReaLTrendingLightning-Fast RL for LLM Reasoning and Agents. Made Simple & Flexible.LLM Infra★ 5.7k—2026-09-10
gym-pybullet-drones logo
gym-pybullet-dronesTrendingPyBullet Gymnasium environments for single and multi-agent reinforcement learning of quadcopter controlLLM Infra★ 2.1k—2026-09-06
llama-cpp-agent logo
llama-cpp-agentTrendingHidden GemThe llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM models, execute structured function calls and get structured output. Works also with models not fine-tuned to JSON output and function calls.LLM Infra★ 660—2026-03-09
OGAM logo
OGAMTrendingThe Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac — GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.LLM Infra★ 3.1kMIT2026-09-10
LLM-VM logo
LLM-VMTrendingirresponsible innovation. Try now at https://chat.dev/LLM Infra★ 490—2024-05-14
jupyter-mcp-server logo
jupyter-mcp-serverTrendingEssential🪐 🔧 Model Context Protocol (MCP) Server for Jupyter.LLM Infra★ 1.3kBSD-3-Clause2026-09-10
wassette logo
wassetteTrendingEssentialWassette: A security-oriented runtime that runs WebAssembly Components via MCPLLM Infra★ 942MIT2026-09-09
tunnel-client logo
tunnel-clientEssentialHidden GemCustomer-run client for Secure MCP Tunnel: connect private or localhost MCP servers to ChatGPT, Codex, the Responses API, and AgentKit without exposing them to the public internet.LLM Infra★ 375Apache-2.02026-09-10
best-of-Agent-Harnesses logo
best-of-Agent-HarnessesTrending🏆 Curated, ranked list of AI agent harnesses (100+) — plus an MCP server, llms.txt & JSON so agents can recommend them too. Rescored weekly.LLM Infra★ 829CC-BY-SA-4.02026-09-09
emisar logo
emisarEssentialHidden GemAn MCP that lets AI tools securely connect to your infrastructure, write IaaS code, debug issues, and assist during incidents - without risking production stability. Built for security teams to approve and infrastructure teams to experience like magic.LLM Infra★ 354NOASSERTION2026-09-10
conduit logo
conduitHidden GemEssentialLocal MCP gateway that cuts your agent's tool-token overhead ~90%. Every MCP server's tools collapse to 3 meta-tools the agent searches on demand. Measured: 97% less per request, same results. Keys in your OS keychain, no cloud.LLM Infra★ 211MIT2026-09-10
1–20 / 68