AMI Labs open-sources H-JEPA: a hierarchical JEPA world model lifts Visual AntMaze planning success from 18% to 73%

H-JEPA is the first public research release from AMI Labs (Advanced Machine Intelligence), the world-model startup associated with Yann LeCun, co-authored with NYU, INRIA Paris and Brown University (arXiv 2610.06805).

Problem. Existing task-agnostic JEPA world models (LeWM, V-JEPA 2, DINO-WM) predict and plan in one latent space at one timescale, so long horizons compound prediction error and blow up the action search, and a single latent must serve both fast low-level dynamics and abstract goal matching.

Method. H-JEPA trains a stack of action-conditioned JEPAs end-to-end. Each level has its own state encoder, action encoder and causal-transformer predictor, reads the level below at stride 2, and is regularized with SIGReg against collapse; upper-level gradients flow into lower encoders. Planning runs top-down: the top level optimizes toward the goal, and its first predicted state becomes the subgoal for the level below, down to primitive actions. Where data has separated timescales, upper levels discard fast detail (leg pose) and keep slow state (position). On real-robot DROID video an inverse-dynamics loss is required to avoid slow-feature collapse.

Results (paper-reported, 3 seeds, not independently replicated). Visual AntMaze: 73.3% success for three levels vs 18.0% for flat LeWM at lower planner compute, and nearly 2x the shared-latent HWM baseline. FourRoom Distractors: 96.0% vs 40.7%. Three levels fail on Push-T because short episodes leave only 58% of training clips. On DROID, H-JEPA improves offline Frechet planning fidelity over HWM and LeWM+IDM at lower budget.

Try it. MIT-licensed code at github.com/kevinghst/H-JEPA (built on LeWM and stable-worldmodel); python main_hjepa.py --config-name ant_hjepa_l3 seed=42. 93 checkpoints (~12.4 GB) on Hugging Face (jepa-world-models/h-jepa). The authors note closed-loop control on a physical robot is the next step.