An adolescent learns to drive in about 20 hours of practice. A self-driving system, Yann LeCun writes, may need thousands of reinforcement-learning trials (training by reward and penalty) to learn that taking a curve too fast ends badly. The teenager never has to skid off the road to find out, because she already has a rough sense of what happens to a moving object that turns too sharply.
LeCun, then at Meta's Fundamental AI Research lab and New York University, built a research program on that gap. His June 2022 paper "A Path Towards Autonomous Machine Intelligence" argues that current AI systems lack an internal model of how the world works, and it sketches an architecture that would learn one. The paper matters here because it states in one place what its author believes language models cannot do and what he would build instead. It is a proposal, not a result: LeCun calls it a position paper rather than a scholarly one, and the text reports no experiments.
How information moves through the architecture
Six modules, one loop. The slow mode runs the loop; the fast mode skips it.
- The actor proposes a sequence of actions.
- The world model predicts the states those actions would produce.
- The cost module scores the predicted states. The gradient of that score tells the actor how to change the actions. Steps 1 to 3 repeat until the score stops improving.
- The agent carries out only the first action, observes the result and starts over.
Mode 1 skips the loop. The policy maps the current state straight to an action, the dashed path in the diagram.
- Information flow
- Mode 1 shortcut
- Storage and retrieval
- Configured by the configurator
Based on section 3 of Yann LeCun, "A Path Towards Autonomous Machine Intelligence" (Version 0.9.2, 2022). Layout and labels are this article's own.
A world model lets an agent test an action before taking it
LeCun describes common sense as a collection of world models, internal models that tell an agent what is likely, plausible or impossible. An agent that has one can simulate the consequences of an action and discard the dangerous ones before the world punishes it.
His architecture has six modules: perception, a world model, a cost module, an actor, a short-term memory and a configurator that sets the system up for the task at hand. In the fast mode, the actor maps what it perceives straight to an action, which LeCun compares to Daniel Kahneman's System 1, the quick and intuitive kind of thinking. The slow mode is the more interesting one. The actor proposes a sequence of actions, the world model predicts the states they would produce, and the cost module scores them. The actor then adjusts the sequence to lower the score, carries out the first action and starts over. Control engineers have used this loop, called model-predictive control, since the 1950s.
The cost module is where goals live. One part is hard-wired and cannot be trained: it computes a single number for the agent's "discomfort," standing in for pain, hunger and similar drives. A second part, a trainable critic, learns to predict future discomfort. LeCun states that the design of the hard-wired part determines how the agent behaves.
Newsletter
Stay up to date on AI.
Clear updates on what AI can do right now and how it affects your work. Every new piece lands in your inbox the day it's published. Free, and you can unsubscribe anytime.
JEPA predicts abstractions and leaves the details out
The world is only partly predictable, and LeCun treats that as the hardest problem in building a world model. Many futures follow from the same present. A system that generates a full prediction, down to every pixel of the next video frame, has to commit to details nobody could know.
His answer is the Joint Embedding Predictive Architecture, or JEPA. Two encoders turn the observed input x and the target y into representations, compact summaries in vector form. A predictor estimates the representation of y from the representation of x, and the prediction error, measured in representation space, serves as the energy: low when the two fit, high when they do not. Because the encoder of y may discard what cannot be predicted, the model never has to reproduce it. LeCun's example is a video of a car approaching a fork in the road. The representations keep position, orientation and speed, and they drop the trees and the texture of the sidewalk. A latent variable, a hidden switch that carries information x lacks, selects whether the car takes the left or the right branch.
The price is stated openly. A JEPA cannot easily generate y; it can only judge whether a y is compatible with a given x.
Technical aside
Why training can collapse (§4.5)
If both encoders output a constant, the prediction error drops to zero and the model has learned nothing. LeCun calls this collapse. Contrastive training, which uses mismatched pairs as negative examples, avoids it, but he argues it becomes inefficient in high dimensions.
In 2022 he proposed four criteria instead: each representation should carry as much information as possible about its input, the representation of y should be predictable from that of x, and the latent variable should carry as little information as possible. The method he described in detail, VICReg (Bardes et al., 2021), keeps the variance of each representation component above a threshold and pushes the covariance between components toward zero.
Later work replaced that recipe with something simpler. LeJEPA (Balestriero and LeCun, 2025) uses a single regularizer, SIGReg, which pulls the distribution of the representations toward an isotropic Gaussian. The authors prove that this rules out collapse, and the method needs only one trade-off setting instead of a stack of training tricks.
Stacked JEPAs plan at several time scales
Since each JEPA drops what it cannot predict, JEPAs can be stacked. The first level makes short-term predictions from low-level representations. A second level takes those representations as input and predicts further ahead at a coarser grain. LeCun's example is a commute. "Get to work" breaks into driving to the station and catching a train, driving breaks into walking out and starting the car, and the chain runs down to muscle control, which can only be filled in once the local conditions, such as obstacles and traffic lights, are perceived. Hierarchical planning usually needs a hand-defined vocabulary of actions. LeCun wants the intermediate levels learned.
Go deeper
The articles argue the position. The courses build the skill.
Learning material built with practitioners, structured chapter by chapter, for readers who want to work through a topic instead of reading about it once.
Later work closed some of the open parts, not all of them
The architecture had gaps of its own, and the paper's section 8.1 lists them. Some have since been tested. In June 2025, Meta's research lab released V-JEPA 2, a 1.2-billion-parameter JEPA trained on internet video. A variant that also takes actions as input, V-JEPA 2-AC, was post-trained on less than 62 hours of unlabeled robot video and then picked up and placed objects with robot arms in two different labs, with no task-specific training and only an image of the goal. That is the slow loop from the diagram running on hardware, with two differences: it searches for actions by sampling candidates rather than following a gradient, and it needs about 16 seconds per action. In March 2026, LeWorldModel showed that a small JEPA can be trained stably end to end from raw pixels on a single GPU, and it planned up to 48 times faster than a comparable world model built on a pretrained encoder.
Other gaps remain open. V-JEPA 2 plans at a single level and over short horizons. Its authors report that errors accumulate as predictions are chained forward, and they name long-horizon planning as future work. The learned intermediate levels of the hierarchy LeCun describes have not been shown. In both systems a human supplies the goal as an image, and nothing yet plays the configurator's role of breaking a task into subgoals. LeCun left Meta in November 2025 and now pursues the program at AMI Labs in Paris, which raised $1.03 billion in March 2026 to build JEPA world models.
Sources and further reading
- Position paper: Yann LeCun, A Path Towards Autonomous Machine Intelligence, Version 0.9.2, June 27, 2022. Section numbers in parentheses refer to this paper.
- Video world model and robot planning: Assran et al., V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning, 2025.
- Collapse-free training: Randall Balestriero and Yann LeCun, LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics, 2025.
- End-to-end training from pixels: Maes et al., LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels, 2026.
- AMI Labs: Report on the March 2026 seed round, VKTR.