InternLM 发布 35B 基础模型 Intern-S2-Mobius

InternLM; Intern-S2-Mobius

Repository navigation

NotificationsYou must be signed in to change notification settings; Fork1 (1); Star59 (59)You must be signed in to star a repository

Repository files navigation

We introduce Intern-S2-Mobius, a 35B foundation model built on the Mobius-v0 architecture realized by Xtuner and LMDeploy. Instead of binding knowledge storage and reasoning computation layer by layer as in conventional Transformer models, Mobius organizes knowledge into a globally shared Memory and lets multiple Reasoners iteratively query and refine hidden states against this shared repository.

This knowledge-reasoning separation gives Intern-S2-Mobius two native capabilities: Backward Residual Connection, where reasoning stages can access knowledge beyond their local layer hierarchy, and Dynamic Latent Reasoning, where deliberation, refinement, and multi-token prediction are internalized into high-density continuous states. Continual-pretrained from Qwen3.5-35B and further post-trained with SFT and RL, Intern-S2-Mobius preserves strong downstream capability while achieving substantially higher end-to-end inference efficiency, with nearly 4x speedup reported in the technical report.

Features

Knowledge-reasoning decoupled architecture. Intern-S2-Mobius separates knowledge vectors from reasoning operators by replacing layer-bound FFN knowledge storage with a globally shared Memory. This gives each Reasoner access to a broader knowledge space and improves knowledge compression compared with a standard Transformer layout; Backward Residual Connection. Through shared Memory, shallow and deep reasoning stages can access knowledge across the model rather than relying only on forward layer-wise information flow. This enables more flexible cross-layer knowledge composition and helps the model synthesize useful information in fewer reasoning steps; Dynamic Latent Reasoning. Mobius refines continuous hidden states through recurrent latent iteration before decoding. This internalizes part of the deliberation process, reduces reliance on long visible chain-of-thought, and dynamically allocates computation to different tokens; Higher inference efficiency with concise reasoning. On reasoning benchmarks, Intern-S2-Mobius reaches comparable or stronger scores than the Qwen3.5-35B baseline while producing markedly shorter reasoning traces and higher request throughput, leading to nearly 4x end-to-end inference speedup in the reported evaluation; Strong general and scientific performance. Intern-S2-Mobius improves the reported average score over Qwen3.5-35B on general reasoning benchmarks, and shows large gains on scientific tasks such as Biology-Instructions, Mol-Instructions, and MolecularIQ

Performance

We evaluate the Intern-S2-Mobius on various benchmarks, including general datasets and scientific datasets. We report the performance comparison with Qwen3.5-35B below. We use the OpenCompass to evaluate all models. For text benchmarks, Intern-S2-Mobius is evaluated with a maximum inference length of 64K tokens on MMLU Pro, SimpleQA, and HLE, and 128K tokens on the remaining text benchmarks.

Quick Start

The Intern-S2-Mobius release is a 35B model stored in bfloat16 weight format. This guide provides deployment examples for the following configurations:

MTP speculative decoding (Recommended); Basic serving without MTP

NOTE: The commands below are reference configurations. Inference frameworks are under active development, so use the latest framework documentation and your local validation results when tuning production deployments.

Intern-S2-Mobius can be deployed using any of the following LLM inference frameworks:

LMDeploy; Transformers; vLLM; SGLang

Sampling Parameters

We recommend using the following hyperparameters to ensure better results

LMDeploy

Use the latest LMDeploy with Intern-S2-Mobius support. The examples below use single-GPU serving.

Serving With MTP (Recommended)

Basic Serving Without MTP

Transformers

Use a recent Transformers version with remote-code loading enabled.

Basic Inference

vLLM

Use the latest vLLM Docker image or source build with Intern-S2-Mobius support.

Serving With MTP (Recommended)

Basic Serving Without MTP

SGLang

Use the lmsysorg/sglang:dev Docker image or a recent source build with Intern-S2-Mobius support. See the SGLang cookbook for Docker commands, verified deployment recipes, benchmarks, and usage examples.

Serving With MTP (Recommended)