AgentIndex icon
AgentIndex
AReaL logo
AReaL
★ 5.9k
vs
on-policy logo
on-policy
★ 2.1k

AReaL vs on-policy

AReaL: AReaL is an open-source, fully asynchronous reinforcement learning training system designed for large reasoning and agentic models. It offers exceptional flexibility, industry-leading speed, and scalability from a single node to over 1,000 GPUs, achieving state-of-the-art performance.; on-policy: This repository implements MAPPO, a multi-agent variant of PPO, widely used in cooperative multi-agent games and research. It provides robust implementations for various multi-agent environments like StarCraft II, Hanabi, and Google Research Football, along with detailed training scripts and hyperparameter guidance.

01

TL;DR

AReaL logo
AReaLLLM Infra · ★ 5.9k

Training Reasoning Agents: Developing AI agents capable of complex mathematical, coding, and general reasoning tasks.

Read AReaL full review →
on-policy logo
on-policyLLM Infra · ★ 2.1k

Research and experimentation in cooperative multi-agent reinforcement learning

Read on-policy full review →
02

Side-by-Side Comparison

Field
AReaL logoAReaL
on-policy logoon-policy
Category
LLM Infra
LLM Infra
Stars
★ 5.9k
★ 2.1k
License
Apache-2.0
MIT
Install Command
pip install -e ".[dev,docs]"
pip install -e .
Updated
1d ago
2y ago
Open Source
Yes
Yes
Website
↗ Visit
↗ Visit
GitHub
↗ GitHub
↗ GitHub
Tags
Reinforcement Learning, Large Language Models, Asynchronous Systems
Multi-Agent Reinforcement Learning, PPO, MAPPO
03

Features

AReaL logoAReaL
01Fully Asynchronous RL Training: Enables stable, industry-leading speed for reinforcement learning.
02Scalability: Seamlessly adapts from single-node setups to over 1,000 GPUs.
03Flexible Agentic Rollout: Easy customization for multi-turn agentic workflows and integration with external frameworks.
04Cutting-Edge Performance: Achieves state-of-the-art results for math, coding, and search agents.
05Open-Source & Reproducible: Provides full training details, data, and infrastructure to reproduce results.
on-policy logoon-policy
01Implementation of MAPPO (Multi-Agent PPO)
02Support for diverse multi-agent environments (e.g., StarCraft II, Hanabi)
03Ready-to-use training scripts for various scenarios
04Detailed hyperparameter guidance and updated results
05Default support for shared policy among agents
04

Use Cases

AReaL logoAReaL
↳Training Reasoning Agents: Developing AI agents capable of complex mathematical, coding, and general reasoning tasks.
↳Large Language Model Alignment (RLHF): Fine-tuning LLMs using Reinforcement Learning from Human Feedback.
↳Multi-Turn Agentic Workflows: Implementing and customizing iterative agent behaviors with self-correction and tool integration.
on-policy logoon-policy
↳Research and experimentation in cooperative multi-agent reinforcement learning
↳Benchmarking and evaluating PPO's effectiveness in MARL scenarios
↳Training AI agents for popular multi-agent games like StarCraft II and Hanabi
05

Best For

AReaL logoAReaL
Trending
on-policy logoon-policy
—
FAQ

FAQ

What is the difference between AReaL and on-policy?
Both AReaL and on-policy are in the LLM Infra category. AReaL has 5.9k stars, while on-policy has 2.1k stars.
Which is better, AReaL or on-policy?
The best choice depends on your use case. Choose AReaL if Training Reasoning Agents: Developing AI agents capable of complex mathematical, coding, and general reasoning tasks., and on-policy if Research and experimentation in cooperative multi-agent reinforcement learning.
Is AReaL free or open source?
Yes, AReaL is open source on GitHub (Apache-2.0).
Is on-policy free or open source?
Yes, on-policy is open source on GitHub (MIT).
Cloud Infrastructure
Ad

Deploy on DigitalOcean

Host your MCP servers, Docker containers, and AI agents with $200 free credit over 60 days.

Claim $200 Free Credit
→

Related

Alternatives to AReaL →Alternatives to on-policy →AReaL details →on-policy details →