AgentIndex icon
AgentIndex
ToolsCategoriesTrendingNewCompare
Submit Tool
ToolsCategoriesTrendingNewCompare
Home/
Browse AI Tool Categories/
LLM Infra

LLM Infra (2026)

Infrastructure tools for deploying, serving, fine-tuning, and evaluating large language models at scale.

51 tools

LLM Infra brings together the most relevant AI tools, MCP servers, and developer products for this topic on AgentIndex. It is designed to help you scan the landscape quickly before narrowing options by stars, licenses, freshness, and practical fit.

This page currently includes 51 tools in the category, making it useful for shortlisting open-source options, comparing popular projects, and building an initial evaluation set before deeper product testing.

NameTaglineCategoryStarsLicenseUpdated
© 2026 AgentIndex.app|Built by a 10-year iOS Developer.
QYSGitHubBuy me a coffee ☕

Browse by Category

Code AssistantWorkflow AutomationRAG / Knowledge BaseMulti-AgentBrowser AutomationLLM InfraDev ToolingObservability

Not affiliated with Anthropic, OpenAI or Microsoft.

MetaGPT logo
MetaGPTMost PopularTrending···🌟 The Multi-Agent Framework: First AI Software Company, Towards Natural Language ProgrammingLLM Infra★ 69.4kMIT2026-01-21
cua logo
cuaMost PopularTrending···Open-source infrastructure for Computer-Use Agents. Sandboxes, SDKs, and benchmarks to train and evaluate AI agents that can control full desktops (macOS, Linux, Windows).LLM Infra★ 19.9kMIT2026-07-16
xLAM logo
xLAMTrendingxLAM: A Family of Large Action Models to Empower AI Agent SystemsLLM Infra★ 634APACHE2026-06-02
on-policy logo
on-policyTrendingReinforcement Learning···This is the official implementation of Multi-Agent PPO (MAPPO).LLM Infra★ 2.1kMIT2024-07-18
youtu-agent logo
youtu-agentTrendingA simple yet powerful agent framework that delivers with open-source modelsLLM Infra★ 4.6kNOASSERTION2026-03-21
mini-swe-agent logo
mini-swe-agentTrendingHidden GemThe 100 line AI agent that solves GitHub issues or helps you in your command line. Radically simple, no huge configs, no giant monorepo—but scores >74% on SWE-bench verified!LLM Infra★ 5.8k—2026-07-14
FedML logo
FedMLTrendingEssentialFEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.LLM Infra★ 4.1kApache-2.02025-10-28
chatarena logo
chatarenaTrendingChatArena (or Chat Arena) is a Multi-Agent Language Game Environments for LLMs. The goal is to develop communication and collaboration capabilities of AIs.LLM Infra★ 1.5kAPACHE2025-08-11
AgileRL logo
AgileRLTrendingHidden GemStreamlining reinforcement learning with RLOps. State-of-the-art RL algorithms and tools, with 10x faster training through evolutionary hyperparameter optimization.LLM Infra★ 935—2026-07-16
AReaL logo
AReaLTrendingLightning-Fast RL for LLM Reasoning and Agents. Made Simple & Flexible.LLM Infra★ 5.6k—2026-07-16
gym-pybullet-drones logo
gym-pybullet-dronesTrendingPyBullet Gymnasium environments for single and multi-agent reinforcement learning of quadcopter controlLLM Infra★ 2.1k—2026-07-11
llama-cpp-agent logo
llama-cpp-agentTrendingHidden GemThe llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM models, execute structured function calls and get structured output. Works also with models not fine-tuned to JSON output and function calls.LLM Infra★ 651—2026-03-09
LLM-VM logo
LLM-VMTrendingirresponsible innovation. Try now at https://chat.dev/LLM Infra★ 491—2024-05-14
best-of-Agent-Harnesses logo
best-of-Agent-HarnessesTrending🏆 Curated, ranked list of AI agent harnesses (100+) — plus an MCP server, llms.txt & JSON so agents can recommend them too. Rescored weekly.LLM Infra★ 320CC-BY-SA-4.02026-07-16
gpu-mcp-server logo
gpu-mcp-serverMCP server for NVIDIA GPU metrics give AI agents real-time access to GPU utilization, memory, temperature, and powerLLM Infra★ 14Apache-2.02026-07-14
conduit logo
conduitLocal MCP gateway that cuts your agent's tool-token overhead ~90%. Every MCP server's tools collapse to 3 meta-tools the agent searches on demand. Measured: 97% less per request, same results. Keys in your OS keychain, no cloud.LLM Infra★ 91MIT2026-07-16
freellmpool logo
freellmpoolFree LLM API pool: 19 LLM providers cataloged, 235 routes, 355 cataloged chat models, keyless start when available.LLM Infra★ 38MIT2026-07-15
llmtrim logo
llmtrimHidden GemEssentialLocal proxy that compresses your LLM API requests so you pay less, with no change to the answers. Trims wasted tokens from prompts, history, tool output, and code before they're sent: -31% input / -74% output, measured live. Any provider, no extra model calls. Also an MCP server and embeddable library (Rust, Python, Ruby, Kotlin, Swift).LLM Infra★ 167AGPL-3.02026-07-16
os-moda logo
os-modaTrendingEssentialYour server fixes itself at 3am. You sleep. AI-native operating system built on NixOS.LLM Infra★ 110Apache-2.02026-06-24
LLM-Agents-Ecosystem-Handbook logo
LLM-Agents-Ecosystem-HandbookHidden GemOne-stop handbook for building, deploying, and understanding LLM agents with 60+ skeletons, tutorials, ecosystem guides, and evaluation tools.LLM Infra★ 535MIT2026-06-30
1–20 / 51