Randall Balestriero and Yann LeCun use JEPA to prove and improve the Platonic representation hypothesis

Randall Balestriero and Yann LeCun propose the Platonic representation theorem: as long as positive sample pairs are two observations of the same scene (good views), JEPAs trained separately on any number of modalities will learn identical representations differing only by a rotation. They point out that next-token prediction is a JEPA using bad views, which explains why language models can only rely on scale and vision alignment; to achieve equivalent alignment on the same corpus, good views require 42 times fewer training samples than next-token prediction, and a 109M-parameter text JEPA aligns with DINOv2 and CLIP image encoders better than all evaluated language models of up to 72B parameters.