AReaL
★ 5.2k
vs
on-policy
★ 2.0k
AReaL vs on-policy
AReaL: AReaL is an open-source, fully asynchronous reinforcement learning training system designed for large reasoning and agentic models. It offers exceptional flexibility, industry-leading speed, and scalability from a single node to over 1,000 GPUs, achieving state-of-the-art performance.; on-policy: This repository implements MAPPO, a multi-agent variant of PPO, widely used in cooperative multi-agent games and research. It provides robust implementations for various multi-agent environments like StarCraft II, Hanabi, and Google Research Football, along with detailed training scripts and hyperparameter guidance.
01
TL;DR
Choose AReaL if…Training Reasoning Agents: Developing AI agents capable of complex mathematical, coding, and general reasoning tasks.
Choose on-policy if…Research and experimentation in cooperative multi-agent reinforcement learning
02
Side-by-Side Comparison
03
Features
AReaL01Fully Asynchronous RL Training: Enables stable, industry-leading speed for reinforcement learning.
02Scalability: Seamlessly adapts from single-node setups to over 1,000 GPUs.
03Flexible Agentic Rollout: Easy customization for multi-turn agentic workflows and integration with external frameworks.
04Cutting-Edge Performance: Achieves state-of-the-art results for math, coding, and search agents.
05Open-Source & Reproducible: Provides full training details, data, and infrastructure to reproduce results.
on-policy01Implementation of MAPPO (Multi-Agent PPO)
02Support for diverse multi-agent environments (e.g., StarCraft II, Hanabi)
03Ready-to-use training scripts for various scenarios
04Detailed hyperparameter guidance and updated results
05Default support for shared policy among agents
04
Use Cases
AReaL↳Training Reasoning Agents: Developing AI agents capable of complex mathematical, coding, and general reasoning tasks.
↳Large Language Model Alignment (RLHF): Fine-tuning LLMs using Reinforcement Learning from Human Feedback.
↳Multi-Turn Agentic Workflows: Implementing and customizing iterative agent behaviors with self-correction and tool integration.
on-policy↳Research and experimentation in cooperative multi-agent reinforcement learning
↳Benchmarking and evaluating PPO's effectiveness in MARL scenarios
↳Training AI agents for popular multi-agent games like StarCraft II and Hanabi
05
Best For
AReaLTrending
on-policyTrendingReinforcement LearningMulti-Agent AI
FAQ
FAQ
What is the difference between AReaL and on-policy?
Both AReaL and on-policy are in the LLM Infra category. AReaL has 5.2k stars, while on-policy has 2.0k stars.
Which is better, AReaL or on-policy?
The best choice depends on your use case. Choose AReaL if Training Reasoning Agents: Developing AI agents capable of complex mathematical, coding, and general reasoning tasks., and on-policy if Research and experimentation in cooperative multi-agent reinforcement learning.
Is AReaL free or open source?
Yes, AReaL is open source on GitHub.
Is on-policy free or open source?
Yes, on-policy is open source on GitHub (MIT).
→