InternLM 开源 SpatialASM：将主动空间心智化内化到 LVLM 的空间推理框架

InternLM; SpatialASM

Repository navigation

NotificationsYou must be signed in to change notification settings; Fork0 (0); Star21 (21)You must be signed in to star a repository

An official implementation of "SpatialASM: Internalizing Active Mentalization for Spatial Reasoning"

Repository files navigation

README

Yuhong Liu* · Shenxi Wu* · Junyi Chi . Beichen Zhang

Xuanlang Dai · Jiaqi Wang† . Yuhang Zang† · Dahua Lin†

*Equal contribution. †Corresponding authors.

🚀 [2026/10/12] We have released the SpatialASM 📖Arxiv paper and its repository; 🚀 [2025/10/12] We have released our allocentric novel-view generator 🤗SpatialASM-NVG; 🚀 [2025/10/12] We have released 🤗SpatialASM-edit140k, an image-text-to-image dataset for tuning novel-view generators, and two spatial-VQA datasets with high-quality CoT responses: 🤗SpatialASM-128k and its subset 🤗SpatialASM-66k

🌈 Overview

Humans rarely solve difficult spatial problems by passively inspecting the views in front of them. Instead, we revisit a scene from task-relevant viewpoints and mentally reconstruct what should be visible there. We call this capability Active Spatial Mentalization (ASM).

We are thrilled to introduce SpatialASM, a scalable framework that internalizes ASM into LVLMs. Through the two-stage training on LVLMs, SpatialASM elicits active spatial exploration and imagination and enhances the overall spatial reasoning performance.

SpatialASM consists of three complementary components:

Allocentric Novel-View Generator. SpatialASM-NVG renders the scene from a target viewpoint specified by an anchor image, a pixel location, and a viewing direction. This allocentric formulation avoids lengthy sequences of camera-centric movements; ASM Trajectory Synthesis. Given a spatial QA sample, an LVLM plans informative viewpoints and the NVG renders the corresponding observations. After GeoP and answer-consistency filtering, rendered views are replaced by textual captions to form explicit plan-imagine-reason trajectories for SFT; View-Imagination Reinforcement Learning. Geometry-filtered candidate views are used to construct automatically verifiable perspective-taking tasks. GRPO further strengthens the association between a viewpoint description and its expected observation

💡 Highlights

🔥 Active spatial reasoning: SpatialASM jointly internalizes task-conditioned view planning and viewpoint-conditioned view imagination, enabling LVLMs to spontaneously seek and mentally construct missing spatial evidence; 🔥 Scalable spatial CoT supervision: SpatialASM converts existing answer-only spatial QA data into explicit plan-imagine-reason trajectories and automatically verifiable perspective-taking tasks, alleviating the scarcity of spatial Chain-of-Thought supervision; 🔥 Training-time spatial simulation: A novel-view generator acts as temporary training scaffolding. The final model does not depend on a world model, renderer, or additional visual observations at inference time; 🔥 Grounded and reliable supervision: An allocentric viewpoint interface reaches a target perspective in one query, while the Geometry Perturbation (GeoP) metric and answer-consistency checks filter unreliable synthesized views and trajectories

📊 Results

We train four LVLM backbones of Qwen3-VL and Qwen3.5 with our SpatialASM framework, and the experimental results on both spatial reasoning and general-VQA tasks are shown below.

⭐️ Quick Start

Here we provide some simple inference code for you to try our models. The models include the LVLMs such as 🤗SpatialASM-Qwen3-VL-4B (see 📢News above), and the novel-view generator 🤗SpatialASM-NVG.

To use the LVLMs, you can either start a vLLM server or load it in your device with Transformers.

🛠️ Training

First download FLUX.2-klein-9B and SpatialASM-edit140k. Then convert the dataset to the format in training_samples.jsonl, and change paths in NVG-training.sh to your local path. And you can start training!

🔥 The training code for LVLMs will be coming soon

🛠️ Evaluation

Prepare the environment:

All benchmarks except ESI-Bench are supported by VLMEvalkit. We use its official repository.

🔥 The evaluation code for ESI-passive will be released soon

🧩Cases

✒️Citation

If you find this project useful, please kindly cite:

📄 License

Use of SpatialASM-NVG: The model SpatialASM-NVG is trained on FLUX.2-klein-9B, so please follow FLUX-NON-COMMERCIAL LICENSE.

Usage and License Notices: The data and code are intended and licensed for research use only.

❤️ Acknowledgement

The evaluation part is built on VLMEvalkit, a powerful toolkit to evaluate a vast range of LMMs!

The NVG training part is built on DiffSynth-Studio, an excellent codebase for Diffusion model training!

An official implementation of "SpatialASM: Internalizing Active Mentalization for Spatial Reasoning"