Modal said in an October 19 blog post that it will add NVIDIA's Arm-based Vera CPU to power Modal Sandboxes, its first step beyond an all-x86 fleet. Using Agentless Agent Bench, a 50-task suite built from Terminal-Bench, Modal measured up to 2x the task throughput of a comparable x86 GNR machine under heavy concurrency, with per-task latency equal or lower (vendor figures). On an idle host running tasks serially, Vera's edge was small. The capacity targets background coding agents and reinforcement learning rollouts.
Key Takeaways
- ✓Throughput: under heavy concurrency Vera reached up to 2x the task throughput of a comparable x86 GNR machine, with per-task latency equal or lower (Modal early tests, vendor figures).
- ✓Benchmark: Agentless Agent Bench uses 50 Terminal-Bench oracle solutions and verifiers with network calls removed, isolating CPU work such as compilation, grep, git and device emulation.
- ✓Idle serial runs: across the 50 tasks Vera was often but not always faster, and not by a large margin; the gap opens under contention.
- ✓Why: Modal credits Vera's high memory bandwidth for packing more containers per core without throttling; x86 alternatives plateaued earlier as CPU counts rose.
- ✓Rollout: this is Modal's first Arm support and needs infrastructure and image-build changes; no launch date or pricing was announced.
Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Background & Industry Pain Points
Coding agents and reinforcement learning both need many sandboxes, often hundreds of containers or VMs on one cloud host. Modal notes that these workloads mostly wait on an LLM, then spike to 10x baseline CPU on an npm install or a large grep, and their shape is unknown in advance. Modal lets users reserve for P50 usage and rely on burst headroom, which saves money but raises noisy-neighbor risk. For background agents, slow CPU means slow answers; for synchronous RL training, every second a sandbox stalls is a second of idle GPU.
Architecture Highlights
Vera is NVIDIA's Arm-compatible CPU, built for the Vera Rubin GPU platform and also sold as a standalone CPU server. NVIDIA hand-delivered the first Vera server to Modal's San Francisco office last week. Modal says Vera's key strength is holding up under contention: high memory bandwidth lets each core pack more container workloads without throttling.
Authoritative Benchmarks & Measured Results
All figures are Modal's own tests (vendor figures); no third-party results were published. Agentless Agent Bench takes 50 Terminal-Bench oracle solutions and verifiers, strips network calls and prewarms them in concurrent Docker containers. In idle serial runs Vera was often but not always faster, by a small margin. In the concurrency test Vera reached up to 2x the throughput of a comparable x86 GNR machine with equal or lower per-task latency; x86 alternatives plateaued as CPU counts rose while Vera kept improving.
Developer Hands-On Guide
Modal says it will roll out Vera support and work with NVIDIA and cloud providers to scale the fleet, but gave no date or pricing. For now, keep using Modal Sandboxes as documented; once Vera lands, check Arm image compatibility, such as x86-only binaries in dependencies. Teams running RL rollouts or large fleets of background agents stand to gain most, since the benefit shows up under high concurrency.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.