Cerebras 在 NeurIPS 2025 发布九篇论文，覆盖预训练到推理

Dec 04 2025

Cerebras is excited to be at NeurIPS 2025—and what a year it's been. We launched our inference API last August, opened a new data center in Oklahoma City, and watched demand for Cerebras Inference explode with the latest state-of-the-art open weight models.

Our research team has been hard at work too, and this year they're presenting nine papers probing the foundational questions of modern AI practice: where does compute get wasted during training? How should reasoning models allocate tokens at inference? When do smaller models beat bigger ones? The work spans pretraining to inference—new findings on scaling laws, training efficiency, and smarter orchestration of test-time compute.

Below is an overview of each paper, what we found, and who should care. Links to the full arXiv papers are included. If you're at the conference, stop by our booth and take a selfie with the wafer. If you're not, we'd still love to hear your thoughts via X or email.

CODA (Conductor-Driven Architecture) optimizes test-time compute across planning, execution, self-refinement, and verification based on problem difficulty—helping 32B models outperform 235B+ competitors in our benchmarks. The insight is that most test-time compute gets wasted: external orchestration and internal model reasoning both try to handle high-level thinking, duplicating effort and burning tokens. CODA separates the roles cleanly—the Conductor plans, reflects, and verifies; the Engine executes—unlocking frontier-level performance without frontier-level model sizes.

Key findings

Who this is useful for

Teams deploying reasoning systems on memory-constrained hardware. Researchers exploring test-time compute scaling. Anyone wanting frontier-level performance without frontier-level model sizes.

📄 [ Paper Link ]

When scaling reasoning models at test time, practitioners face a frustrating bottleneck: these models are terrible at evaluating their own solutions. Multi-path exploration strategies like best-of-n sampling rely on the model knowing when it's right or wrong, but current reasoning models are heavily biased toward giving everything high scores regardless of correctness. This paper introduces an Explanatory Verifier trained via reinforcement learning (GRPO) that produces calibrated confidence scores along with natural language reasoning for why solutions are correct or incorrect. The key insight is comparing pairs of candidate solutions rather than evaluating them in isolation, which helps the verifier catch subtle errors.

Key findings

Who this is useful for

Teams running best-of-n or majority voting at scale and burning tokens on redundant generations. Researchers working on reward models and verifiers for reasoning. Anyone who's noticed their reasoning model is confidently wrong and wants a better filter.

📄 [ Paper Link ]

Speculative decoding has become a go-to technique for accelerating text-only LLMs, but applying it to vision-language models is harder—visual and textual information need to stay tightly integrated throughout generation. DREAM is a speculative decoding framework built specifically for VLMs, combining three innovations: a cross-attention mechanism that injects intermediate features from the target model into the draft model, entropy-adaptive feature selection to guide draft training, and visual token compression to reduce latency without losing critical information.

Key findings

Who this is useful for

Teams deploying vision-language models in latency-sensitive applications. Researchers working on efficient multimodal inference. Anyone running VLMs at scale and looking to cut decoding time without sacrificing output quality.

📄 [ Paper Link ]

When scaling up transformer models, hyperparameters like learning rate that work well for smaller models often need expensive retuning as you increase depth. This paper investigates parameterizations—rules for how to adjust hyperparameters as model architecture changes—and identifies one called CompleteP that enables hyperparameters to transfer reliably across both width and depth. The core change is scaling each residual block's output by 1/L before adding it to the residual stream, which maintains stability while ensuring every layer fully exploits its nonlinear capacity.

Key findings

Who this is useful for

ML engineers training large transformers. Researchers studying neural scaling laws. Teams with hardware constraints favoring narrow-deep architectures.

📄 [ Paper Link ]

Hyperparameters like weight decay and batch size that work at small scale need expensive retuning as model size, dataset size, and batch size change. This paper investigates scaling laws for these hyperparameters and identifies precise power-law relationships that enable accurate prediction of optimal settings before large-scale training begins.

The core insight is modeling training through what the authors call the AdamW timescale—a ratio that captures how weight updates are integrated over training, combining batch size, learning rate, weight decay, and dataset size into a single quantity. Rather than tuning learning rate as batch size changes (which hits stability limits), tuning weight decay maintains optimal timescale across a much wider range.

Key findings

Who this is useful for

ML engineers setting hyperparameters for large-scale pretraining. Teams making compute/time tradeoffs in model training. Researchers studying neural scaling laws and optimal training configurations.

📄 [ Paper Link ]

When adapting a pre-trained language model to a new domain (like adding French to an English/Arabic model), you need to balance learning the new domain against forgetting the old one. Existing scaling laws for this process assume a fixed pre-training budget, but in practice you're often working with models trained to different extents. This paper introduces scaling laws that explicitly incorporate the pre-training budget (measured as tokens-per-parameter, or PTPP), enabling predictions about adaptation performance for models you haven't actually trained yet.

Key findings

Who this is useful for

Teams adapting foundation models to new languages or domains. Researchers studying continual pre-training dynamics. Anyone trying to plan adaptation compute budgets without running exhaustive experiments.

📄 [ Paper Link ]

Cerebras has a deep collaboration with Inception Institute of Artificial Intelligence on foundational models for life sciences. Together we've published three papers introducing the Omics42 platform: Prot42, a family of protein language models that generates high-affinity protein binders from sequence alone that achieve binding strengths competitive with Google's AlphaProteo while handling sequences up to 8,192 amino acids. Chem42, a family of chemical language models for drug-like molecule generation that outperforms larger models on standard benchmarks with 6x fewer parameters, and can design ligands tailored to specific protein targets. Gene42, a genomic foundation model that processes up to 192,000 DNA base pairs at single-nucleotide resolution—the first dense-attention model to achieve this scale—setting state-of-the-art on species classification and disease variant prediction tasks.

📄 Chem42: a Family of Chemical Language Models for Target-aware Ligand Generation

Performance comparisons are based on third-party benchmarking or internal testing. Observed inference speed improvements versus GPU-based systems may vary depending on workload, configuration, date and models being tested.

1237 E. Arques Ave Sunnyvale, CA 94085