Rapid-MLX
Rapid-MLX is an LLM inference server for Apple Silicon built on MLX, with OpenAI-compatible (`/v1/chat/completions`, `/v1/responses`) and Anthropic Messages (`/v1/messages`) APIs and a focus on reliable tool calling for coding agents. Its README claims up to 4x faster decoding than Apple's `mlx-lm` and 1.5x on a typical task for the same model and weights, measured with Qwen3.5-9B 4-bit on a Mac mini M4 Pro; those figures are the project's own measurements. A feature comparison table against oMLX, Ollama, LM Studio and mlx-lm is built from each project's documentation and dated 2026-09-26. It has continuous batching, a radix prefix cache saved to disk and restored at startup, and 27 tool-call parser modules, and it also covers image, video and audio generation, transcription and voice cloning.
Install on an M-series Mac with `brew install rapid-mlx` or the guided installer (`curl -fsSL https://rapidmlx.com/install.sh | bash`), which detects your RAM and recommends a starter model. Then `rapid-mlx chat` starts a REPL (first run downloads about 3 GB of weights) and `rapid-mlx serve qwen3.5-4b-4bit` serves an endpoint on the first free port from 8000 to 8009. `rapid-mlx launch claude-code` patches Claude Code's local config to route to the local server. There is also a signed macOS desktop app, a community benchmark leaderboard you can opt into, and a tested compatibility matrix for five agents. The README says it is Apache-2.0, while GitHub's license detection shows none, so check the repository.
Features
Why choose it
Trade-offs
Maintenance check
- Last commit
- 2026-10-22
- Commits, last 90 days
- 100+
- Releases, last 90 days
- 30
- Latest release
- v0.16.8 (2026-10-22)
- Open issues
- 35
- Recent issues with at least one comment
- 2/4
Oldest week on the left; the latest four weeks are orange. Source: GitHub commit activity.
Status rule: archived on GitHub = Archived; last commit within 90 days = Active; 91 to 365 days = Slowing; older = Stale. All figures come from the GitHub API on the date shown.
Most discussed open issues
- #4451 Investigate warm hybrid repeats displacing reusable boundaries under cache pressure3 comments · 2026-10-10
- #5083 [Vector] Quantized KV cache with attention-sink / sliding-window support (Flint)2 comments · 2026-10-22
- #5020 DeepSeek-V4.1 batched decode lanes run well below single-stream throughput2 comments · 2026-10-21
- #4857 Desktop UX: Lead first setup with one understandable starter instead of an overlapping model catalogue2 comments · 2026-10-18
- #5038 DeepSeek V4.1: DSpark draft gives no net speed-up on real chat requests (break-even acceptance)1 comments · 2026-10-21
Issue titles are quoted from GitHub. Open the link to check whether a problem still applies to the version you use.
Compatibility
Quick start
Use cases
Alternatives
Comments
No comments yet. Be the first!