Dongxi recommends a paper on generative representation learning
English text is being prepared.
Dongxi NLP recommends reading the paper "Generative Representation Learning for Scalable Latent Tokenization." As introduced in the quoted post, this work proposes a diffusable representation learning method that remains scalable even when DINO/LeJEPA fail, and presents a joint scaling law for the tokenizer and diffusion model that is more compute-optimal than raw diffusion.
Topics
View original post1/ The tokenizer is a new scaling axis! We introduce an approach for diffusable representation learning that's principled, scalable, and works when DINO/LeJEPA collapse. We get joint tokenizer+diffusion scaling laws and it's compute-optimal vs raw diffusion w/ @drfeifei @GordonWetzstein @jcjohnss @ericryanchan @vincesitzmann @KyleSargentAI Thread, Paper, and Blogpost below
View original
1/ The tokenizer is a new scaling axis! We introduce an approach for diffusable representation learning that's principled, scalable, and works when DINO/LeJEPA collapse. We get joint tokenizer+diffusion scaling laws and it's compute-optimal vs raw diffusion w/ @drfeifei @GordonWetzstein @jcjohnss @ericryanchan @vincesitzmann @KyleSargentAI Thread, Paper, and Blogpost below
Body
推荐阅读！