verl-agent

verl-agent extends veRL for training LLM agents with reinforcement learning. The README contrasts it with approaches that concatenate full interaction histories: it uses a step-independent multi-turn rollout, so each step's input structure, history management and memory module can be customised, which it says helps long-horizon tasks. It accompanies the GiGPO (Group-in-Group Policy Optimization) paper, accepted at NeurIPS 2025.

The feature table lists multi-turn and step-wise agent-environment interaction, a customisable memory module, parallelised Gym environments, text-only and text-plus-image modalities, and LoRA training. Models include Qwen3, Qwen3-VL, Qwen2.5, Qwen2.5-VL and LLaMA3.2. Algorithms include GiGPO, GRPO, PPO, DAPO, GSPO, RLOO and REINFORCE++. Environments include ALFWorld, WebShop, Search (tool calling), Sokoban, Gym Cards and AppWorld, and there is a GPT-4o prompt-based agent.

Installation is research-grade: create a Python 3.12 conda env, install `vllm==0.11.0` and `flash-attn==2.7.4.post1`, run `pip install -e .`, then install each environment separately, ideally in its own environment. The README news mentions support by the ROLL and OpenManus-RL projects. Licence is Apache 2.0 and the last push was 2026-06-09, with README news running to 2025-12. This review is based on the README and repository data; we have not run it ourselves.