Viggle Releases JST-2, a 3D-Native World Model
Viggle has released JST-2, building its entire model stack in 3D space and representing the world with Physics-Native Tokens that carry xyz coordinates, attribute vectors, and dynamics vectors, rather than implicitly inferring physical laws from video pixels. The architecture consists of three parts: the JST encoder, JST generator, and JST decoder. A single forward inference generates a complete 3D scene in about 1 second, at a cost below $0.01 (B200), and supports up to 64 input views as a spatial context window. Blind human preference tests published by Viggle show that character generation wins 58.4% against Meshy 7.1 and 73.2% against Tripo H3.1, while video-generated motion wins 89.1% against DeepMotion. JST-3 plans to extend the learned dynamics to object interaction.
Why read this
AI score 65
The original article reviews how JST-2 builds physical structure into the representation itself, and outlines how its approach differs from video-space world models.
Topics
View original postToday we are releasing JST-2, the world's first full-stack physics-native world foundation model. Many world models learn in pixel/video space. We went back to first principles and built the entire model stack in 3D. Generate a 3D scene in seconds and expand it in real time. Create characters with motion built in, all through one general architecture.
View original
Today we are releasing JST-2, the world's first full-stack physics-native world foundation model. Many world models learn in pixel/video space. We went back to first principles and built the entire model stack in 3D. Generate a 3D scene in seconds and expand it in real time. Create characters with motion built in, all through one general architecture.
Body · AI translation
Recently, mainstream world models (Sora, Genie, Genie-3, Cosmos...) almost all operate in pixel/video space, where the model implicitly "guesses" physical laws from massive amounts of video.
@ViggleAI's newly released "JST-2" takes the opposite path: the entire tech stack is built directly in 3D space.
This is the key to understanding the whole announcement: physical structure is not inferred from visual appearance; it is part of the representation itself. Architecturally, it is split into three components: the JST encoder (encodes images into physics-native tokens), the JST generator (generates tokens directly from multimodal prompts), and the JST decoder (combines viewpoint and motion/force inputs to decode into images or 3D Gaussian splatting).
Viggle spent four years (JST-1 in 2022–2024, JST-2 in 2024–2026) building the full stack in-house: tokenization, architecture, data synthesis engine, pretraining, and post-training. The in-house data engine matters because 3D-native training data barely exists in nature and must be obtained through synthesis, which is both a moat and a cost.
Capabilities: one architecture unifies three things. · Scene generation: a single forward pass generates a complete 3D scene in about 1 second at a cost of <$0.01 (B200), with no per-scene optimization; supports up to 64 input viewpoints as a "spatial context window," allowing the same world to be continuously extended. · Character generation: generates 3D characters from a single image (cropping/occlusion allowed) or from text, with dynamics learned directly from data, skipping rigging and skinning. · Motion generation: text- or video-driven full-body 3D motion; in video mode, it can reconstruct visible parts and infer occluded body parts. · Scene reconstruction: reconstructs complete 3D environments from sparse real-world observations, pointing toward robot simulation, autonomous driving, and mapping scenarios.
Benchmarks: blind human evaluation, and on the battlefield where opponents excel. Viggle published blind human preference comparisons (numbers are JST-2 win rates): · Character generation: beats Meshy 7.1 (58.4%), Tripo H3.1 (73.2%). · Animation readiness: beats Meshy 73.8%, Tripo 88.9%, with the advantage concentrated in hand rigging and deformation quality. · Text-to-motion: beats Cartwheel Swing 74.8%, HY-Motion-1.0 (presumably Hyper3D) 80.0%. · Video-to-motion: beats Cartwheel 58.1%, DeepMotion 89.1%, with hand tracking win rates as high as 95%.
JST-3's future roadmap: from "3D that moves" to "physics that is learned." In JST-2, character motion is driven by learned dynamics (which is why it skips rigging/skinning), but interactions between objects are still rule-based simulation. JST-3's goal is to generalize learned dynamics to object interaction, replacing rule-based simulation with learned physics.
The ultimate vision is stated clearly: an "experience engine" for physical AI, where agents learn in simulated worlds by acting and observing the consequences. This is essentially an answer to the question "where does robot training data come from," aiming at the same endgame as NVIDIA Cosmos, Genesis, and Google's world model research, but entering from the angle of 3D-native representation rather than video generation.