Mosaic: Multi-Session Agentic Reinforcement Learning for Context Compaction and Subagent Forking

Background and the Problem

Production coding and deep-research agents rely heavily on context compaction (summarizing and resetting context windows) and subagent forking (spawning isolated worker sessions) to scale across long horizons. These techniques split a single task trajectory across multiple discrete context sessions. However, standard agentic RL assumes a single continuously accumulated session, and naively broadcasting trajectory-level rewards equally across all sessions fails to distinguish useful subagents from wasteful ones.

Architecture and How It Works

Researchers introduced Mosaic (arXiv:2610.22174), a critic-free hierarchical optimization framework for Multi-Session Agentic Reinforcement Learning:

1. Trajectory-Level Session-Usage Signals: Integrates session allocation metrics directly into trajectory-level advantage estimation to discourage redundant session spawning;
2. Session-Level Marginal Contribution Shaping: Estimates each individual session's marginal contribution to task completion and shapes session-specific advantages accordingly, without requiring a separate value critic.

Benchmarks and Measured Results

Evaluations on BrowseComp using Qwen3.5-4B demonstrate clear superiority over standard trajectory broadcasting:

• Wins Across Both Compaction and Subagent Forking: Mosaic consistently achieves higher task resolution accuracy than trajectory-level advantage broadcasting in both context-compaction and subagent-forking regimes;
• Higher Efficiency with Fewer Sessions: Agents trained with Mosaic use significantly fewer sessions on average while achieving higher single-Session utility.

Getting Started for Developers

The open-source codebase is available at https://github.com/LARK-AI-Lab/Mosaic:

• Align RL Training with Multi-Session Harnesses: Use Mosaic to train models that natively decide when to invoke context compaction or fork subagents without wasting compute on low-utility branches.