【DL輪読会】Why Accurate Prediction Is Not Enough: Long-Horizon Planning with Latent World Models

DEEP LEARNING JP [DL Papers] Why Accurate Prediction Is Not Enough: Long-Horizon Planning with Latent World Models Jianqi Yang, Matsuo Iwasawa Lab, M1 http://deeplearning.jp/ 1

• http://deeplearning.jp/

Background | 1 · The first recipe (2018–2023) Background DINO-WM Planning Limits HWM Conclusion A world model lets an agent try actions in imagination THE IDEA THE FIRST RECIPE · learn the model from pixels and rewards what it sees what it might do frame observation candidate action pixels Encoder latent Dynamics + action next latent Decoder predicted pixels action World model Reward head learned from experience Behaviour loss 1 match the real pixels pick actions that maximise predicted reward loss 2 what would happen predicted outcome match the real reward 2018 · World Models. Only the small controller C sees the reward. 2019 · PlaNet. Pick actions with the most predicted reward, 12 steps. 2023 · DreamerV3. Reconstruct every pixel (the decoder term). Try actions in imagination; act on the best. WHAT IT SOLVED WHAT IT EXPOSED Control from pixels, by planning or learning inside the model's imagination. The model is graded on every pixel. And each task needs its own reward, so a new task means training again. BACKGROUND World Models (2018) · PlaNet (ICML 2019) · DreamerV3 (2023) · Schematic: mine; PlaNet formula from its description 2

Background | 2 · The JEPA idea (2022–2024) Background DINO-WM Planning Limits HWM Conclusion JEPA: predict the representation, not the pixels (b) Generative: loss on pixels (c) JEPA: loss on representations ŷ is decoded pixels; D compares it with the real y. Every detail must be right, even the unpredictable. The answer comes from a slow copy of the encoder (EMA) with no gradient (sg): no decoder, and no collapse. (a) Not today's topic Two encoders pulled together; no predictor. 2022 · LeCun, position paper Proposes JEPA, and stacked JEPAs that predict at several time scales, as the basis of a world model. BACKGROUND 2023 · I-JEPA → Images: predict the representations of hidden blocks from one visible block. 2024 · V-JEPA → Video: predict the features of masked regions. No pixel reconstruction. → Figure: I-JEPA (CVPR 2023), Fig. 2 · JEPA formula: V-JEPA 2, Eq. 1 · LeCun 2022 (via summaries) · V-JEPA (2024) Still missing No action goes in. These models learn to see, not yet to act. 3

Background | 3 · From JEPA to a planner (2024–2026) Background DINO-WM Planning Limits HWM Conclusion Add actions, swap the reward for a goal image, and search at test time + Actions go into the predictor Reward → a goal image is the task 1 image now Encoder Policy → search at test time 3 latent 4 Predictor predicted latent Cost rolled out K steps after K steps squared distance to the goal latent candidate actions 5 MPC: execute a few actions, observe, plan again. 1 Latent · image → vector 2 Goal image · the task 2 scores Optimiser (CEM) goal image encoded the same way sample, keep best, repeat 3 Rollout K · imagine K steps 4 Cost · distance to goal 5 CEM + MPC · search, act, replan Two knobs for the rest of the talk: how far the rollout reaches (K), and how a plan is scored (the cost). BACKGROUND DINO-WM (ICML 2025) · V-JEPA 2 (arXiv:2506.09985) · PLDM (arXiv:2502.14819) · LeWorldModel (arXiv:2603.19312). Diagram: mine 4

Background | 4 · Where this branch sits Background DINO-WM Planning Limits HWM Conclusion GRADED ON PIXELS GRADED ON REPRESENTATIONS ONLY reconstruct or generate no decoder TODAY Visual Foresight 2018 goal-image planning, in pixel space or none TASK = A GOAL IMAGE This branch drops the pixels and the reward, and pays with horizon drop the pixels Genie 2024 · Cosmos 2025 video generators, proposed as simulators DINO-WM 2024 · PLDM 2025 V-JEPA 2-AC 2025 LeWorldModel 2026 one per task TASK = A REWARD drop the reward latent dynamics, graded on pixels and reward OUR SYNTHESIS Every world model must answer two design questions: what is it trained to predict, and what tells it the task? Two common answers each give four cells. goal-image planning in feature space the JEPA idea (2022–24), plus actions World Models 2018 PlaNet 2019 · DreamerV3 2023 WHY THESE TWO AXES IS IT MECE? Each axis splits cleanly in its main form. Not fully exhaustive: hybrids exist, e.g. language goals, or rewards learned from images. MuZero 2019 · TD-MPC2 2024 drop the decoder no decoder; graded on reward and value Placement: mine, from each paper's own description. Full references on the reference slide WHY THIS CORNER No labels and no pixels: learns from any video with actions, takes any goal image, searches cheaply. Price: no value to see past the horizon. 5

Background | 5 · The open problem Background DINO-WM Planning Limits HWM Conclusion Near goals work, far goals do not: one problem, three papers WHAT IS UNSOLVED THREE PAPERS, THREE PARTS OF THE LOOP Same planner, farther goals (Push-T) HOW TO PLAN AT ALL 84% 1 How can a model trained without rewards solve a new task? DINO-WM ICML 2025 · builds the loop 55% 17% 25 steps 50 steps WHY IT BREAKS 2 Why does planning fail when the goal is far? Planning Limits arXiv 2026 · measures the loop 75 steps HOW TO FIX IT 3 V-JEPA 2 (2025), pick-and-place: boxed frames are sub-goal images given by hand. It names two causes: errors accumulate over the rollout; the search grows exponentially with horizon. Can the model propose its own subgoals? HWM arXiv 2026 · adds a second level HWM builds on DINO-WM. The two 2026 papers do not cite each other; reading them together is my synthesis. Today: why does planning break when the goal is far, and how can it be fixed? AUTHORS' REPORT HWM (arXiv:2604.03208v3), Table 2 · Strip: V-JEPA 2 (arXiv:2506.09985), Fig. 10, first row 6

Paper 1 | DINO-WM (1/3) Background DINO-WM Planning Limits HWM Conclusion DINO-WM: plan in the feature space of a frozen vision encoder Question Can one world model, trained on offline data with no rewards, solve new tasks at test time? Answer Yes, for short tasks: predict DINOv2 patch features, then search for actions that reach a goal image. ROLE The foundation TESTED ON · six environments, 224×224 images NO PIXELS The state is a set of features Each frame becomes the patch features of a frozen DINOv2 encoder. Nothing is reconstructed. NO REWARD The task arrives later, as an image Training uses only offline trajectories of images and actions. No reward, no task label. NO POLICY Behaviour comes from search Actions are optimised at test time, so the same model serves any goal. Top: maze, reach, wall · Bottom: Push-T, rope, granular. Baselines trained on the same reward-free data. AUTHORS' REPORT Zhou et al. DINO-WM. ICML 2025 (arXiv:2411.04983v2) · Right: Fig. 3 7

Paper 1 | DINO-WM (2/3) Background DINO-WM Planning Limits HWM Conclusion How it works: train a latent predictor, then search with it TRAINING · offline trajectories, no reward past frames Encoder latents Predictor latents + actions → next latent predicted next latent squared error features of the real next frame PLANNING · at test time, given a goal image (the paper's Fig. 2) HOW TO READ IT Orange: the actions, optimised at test time. Blue band: everything stays in latent space. Planning loss: last predicted latent vs goal latent. Only p_θ is trained; DINOv2 is frozen. AUTHORS' REPORT Zhou et al. DINO-WM. ICML 2025 (arXiv:2411.04983v2) · Sec. 3 · Bottom: Fig. 2 8

Paper 1 | DINO-WM (3/3) Background DINO-WM Planning Limits HWM Conclusion It beats reward-free baselines on short tasks, and leaves long ones open Push-T: other world models Push-T: other encoders 90% WHAT IT LEAVES OPEN 90% • Authors: needs offline data that covers the states and actions. • Authors: needs ground-truth action labels. • Authors: plans in raw action space; they propose a hierarchy. 42% 32% 44% 30% • Not tested: goals were sampled within 25 steps. 20% 0% IRIS DreamerV3 TD-MPC2 DINO-WM ResNet R3M DINO CLS DINO patch Handed to the next paper: what happens when the goal is far? AUTHORS' REPORT Zhou et al. DINO-WM. ICML 2025 (arXiv:2411.04983v2) · Tables 1–2, Sections 4.1 and 5 9

Paper 2 | Planning Limits (1/2) Background DINO-WM Planning Limits HWM Conclusion Planning Limits: measure how far ahead a planner can rank actions Question When the goal is far, where does planning break: the predictor, the search, or the scoring? Answer A 5-step rollout ranks actions reliably only 5–10 steps ahead. Tasks take 16–53 steps. ROLE The diagnosis 5–10 steps the planner can rank (K = 5) • K, rollout horizon: steps the planner imagines. • L, target lookahead: how far ahead the target frame is. • Plannable range: the largest L where the expert still beats 90% of 100 random action sequences. AUTHORS' REPORT 16–53 steps a task takes Alrasheed, Azam, Akhtar. The Planning Limits of Latent World Models. arXiv:2609.39235v1 (unreviewed) · Figure: Fig. 1 10

Paper 2 | Planning Limits (2/2) Background DINO-WM Planning Limits HWM Conclusion A better predictor does not extend the range; a closer target does HELPED DID NOT HELP Closed-loop success, 16 tasks 92% → 41% 76% Exact simulator, 5-step rollout, as the target moves from 5 to 20 steps ahead (Fig. 1b) 47% 81× larger predictor Same plannable range 23% 30% 10× more samples 0 of 60 before, 0 of 60 after, on three failing tasks No feedback MPC MPC, 30-step rollout, same compute MPC + expert subgoals Handed to the next paper: those subgoals came from an expert. Who supplies them without one? AUTHORS' REPORT Alrasheed, Azam, Akhtar. The Planning Limits of Latent World Models. arXiv:2609.39235v1 (unreviewed) · Fig. 1, Sec. 5–6 11

Paper 3 | HWM (1/4) Background DINO-WM Planning Limits HWM Conclusion HWM: a second, slower world model proposes the subgoal Question Can a planner generate its own subgoals, with no reward and no learned policy? Idea Two world models share one latent space. The slow one plans; its first prediction is the fast one's goal. ROLE The remedy One training signal Both levels learn by predicting the next latent. No reward, no policy. One latent space The subgoal is a latent, not an image or an instruction. Reuses the flat model The fast level is V-JEPA 2-AC, DINO-WM or PLDM. Frames other than the current and goal observations are decoded for display; planning never generates images. AUTHORS' REPORT Zhang et al. Hierarchical Planning with Latent World Models. arXiv:2604.03208v3 · Figure: Fig. 1 (top) 12

Paper 3 | HWM (2/4) Background DINO-WM Planning Limits HWM Conclusion How it works: learn macro-actions, then plan at two speeds TRAINING THE SLOW MODEL (the paper's Fig. 3) PLANNING AT TEST TIME 1 2 3 A chunk of actions between waypoints (25–70 steps apart on Push-T) becomes one macro-action l. The slow model predicts the next waypoint's latent. Loss: L1 in latent space. Slow level: search a few macro-actions so the last predicted latent matches the goal. Take its first predicted latent as the subgoal. Fast level: search a few primitive steps toward the subgoal. Execute, then replan both levels. WHY MACRO-ACTIONS Searching over concatenated raw actions gives 52% and 37% in the maze (medium and long paths). Macro-actions give 95% and 83%. AUTHORS' REPORT Zhang et al. Hierarchical Planning with Latent World Models. arXiv:2604.03208v3 · Sec. 2, Table 4, App. A–B · Left: Fig. 3 13

Paper 3 | HWM (3/4) Background DINO-WM Planning Limits HWM Conclusion Hierarchy recovers success, and the gain grows with distance Push-T (goal distance, steps) Maze (path length) 100% 100% 84% 89% Real Franka, one goal image 84% 78% Cup Box Drawer Flat 0% 0% 30% Hierarchical 70% 60% 70% π0.5 (VLA) 68% 36% – 95% 83% 61% 55% 44% 17% 5 trials per configuration. The VLA baseline receives language goals. 25 50 Flat Hierarchical 75 short medium Flat long Hierarchical On Push-T the gain grows with distance: +5, +23, +44 points. AUTHORS' REPORT Zhang et al. Hierarchical Planning with Latent World Models. arXiv:2604.03208v3 · Tables 1–3 14

Paper 3 | HWM (4/4) Background DINO-WM Planning Limits HWM Conclusion Two controls say it is the hierarchy; four limits remain CONTROL 1 · SAME CAPACITY → NOT MODEL SIZE CONTROL 2 · GIVE IT A SUBGOAL → NOT THE FLAT MODEL Push-T success 78% Franka pick-and-place Cup Box Flat, no subgoal 0% 0% Flat + manual subgoal 80% 80% HWM, its own predicted subgoal 70% 60% HWM + manual subgoal 80% 80% Octo (VLA) + manual subgoal 20% 10% 61% 55% 35% 17% 50 steps 15% 75 steps Flat, 44M Flat, 98M Hierarchical, 94M Limits (authors, Sec. 6). Success still falls with distance. A subgoal can be out of the fast level's reach, and nothing flows back from it. Both levels share one latent space; waypoints follow a fixed rule. AUTHORS' REPORT Zhang et al. Hierarchical Planning with Latent World Models. arXiv:2604.03208v3 · Tables 13, 15, Sec. 6 15

Conclusion | The chain Background DINO-WM Planning Limits HWM Conclusion What each paper settled, and what it handed to the next 1 DINO-WM 2 How to plan at all Planning Limits 3 Why it breaks HWM How to fix it SETTLED SETTLED SETTLED A latent predictor on frozen features can plan to a goal image, with no reward. On far goals the failure is in the planner's reach and scoring, not in prediction accuracy. A second world model can generate the subgoals: 17% → 61% on Push-T, 0% → 70% on a real robot. → → LEFT OPEN LEFT OPEN LEFT OPEN Goals beyond about 25 steps. Where subgoals come from without an expert. Success still falls with distance; subgoals can be out of reach. Together: a short rollout cannot see a far goal, and a long one is hard to search. Subgoals turn one long search into several short ones. OUR SYNTHESIS Synthesis: mine. Evidence: DINO-WM Table 1 · Planning Limits Sec. 5–6 · HWM Tables 1–2, 13, 15 16

Conclusion | What comes next Background DINO-WM Planning Limits HWM Conclusion Where this goes next: research, uses, and neighbouring fields HWM puts the goal back within reach. What it cannot do yet sets the agenda. WHERE HWM GOES NEXT WHERE IT COULD BE USED WHAT IT MEANS NEXT DOOR • A learned “steps-to-go” cost instead of the greedy distance to the goal image. • Robots instructed by a photo: zero-shot manipulation with no per-task reward (VJEPA 2-AC, HWM on a real Franka). • RL: reward-free offline data becomes usable; the missing piece is a goalconditioned value. • Long household and warehouse tasks: exactly the long, non-greedy case. • Video generation (Genie, Cosmos): two ways to build a simulator, pixels or features: realism vs cost. • Reachable subgoals: let the low level report back. Mind the Gap's constrained search is a first step. • When to switch on the hierarchy: read the planner's own score spread (my proposal). • Video understanding: V-JEPA 2's encoder also does action anticipation and video QA. • Learned waypoints; more than two levels. OUR SYNTHESIS Outlook: my view · Evidence: HWM Sec. 6 · V-JEPA 2 (2025) · Mind the Gap (2026) • LLM agents: the same pattern: plan, split into subgoals, decide where test-time compute goes. 17

References Papers presented, and the background they stand on PRESENTED [1] G. Zhou, H. Pan, Y. LeCun, L. Pinto. DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning. ICML 2025. arXiv:2411.04983 [2] A. J. Alrasheed, B. Azam, N. Akhtar. The Planning Limits of Latent World Models. arXiv:2609.39235, 2026 (preprint) [3] W. Zhang, B. Terver, A. Zholus, S. Chitnis, H. Sutaria, et al. Hierarchical Planning with Latent World Models. arXiv:2604.03208, 2026 (preprint) BACKGROUND (SLIDES 2–6) AND FOLLOW-UP [4] D. Ha, J. Schmidhuber. World Models. arXiv:1803.10122, 2018 [5] D. Hafner et al. Learning Latent Dynamics for Planning from Pixels (PlaNet). ICML 2019 [12] L. Maes et al. LeWorldModel: Stable End-to-End JEPA from Pixels. arXiv:2603.19312, 2026 [13] F. Ebert et al. Visual Foresight. arXiv:1812.00568, 2018 [6] D. Hafner et al. Mastering Diverse Domains through World Models (DreamerV3). arXiv:2301.04104, 2023 [14] J. Schrittwieser et al. Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model (MuZero). arXiv:1911.08265 [7] Y. LeCun. A Path Towards Autonomous Machine Intelligence. OpenReview, 2022 [15] N. Hansen, H. Su, X. Wang. TD-MPC2: Scalable, Robust World Models for Continuous Control. ICLR 2024 [8] M. Assran et al. Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture (I-JEPA). CVPR 2023 [16] J. Bruce et al. Genie: Generative Interactive Environments. arXiv:2402.15391, 2024 [9] A. Bardes et al. Revisiting Feature Prediction for Learning Visual Representations from Video (V-JEPA). arXiv:2404.08471, 2024 [17] NVIDIA. Cosmos World Foundation Model Platform for Physical AI. arXiv:2501.03575, 2025 [10] Meta FAIR. V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning. arXiv:2506.09985, 2025 [18] N. Caselli et al. Mind the Gap: Promises and Pitfalls of Hierarchical Planning in LeWorldModel. NeurIPS 2026 Workshop (PTA). arXiv:2607.12547 [11] V. Sobal et al. Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models (PLDM). arXiv:2502.14819, 2025 18