Back to listLing-3.0 tiny & flash Base Models Officially Open-SourcedAugust 20, 20268 min read

Since August, the Ling-3.0 series models have been released one after another. We have successively open-sourced Ling-3.0-flash and Ling-3.0-tiny, hoping to provide developers with open model options covering different scales and different task requirements.

Since the open-sourcing, more and more developers have begun to explore their own application scenarios based on Ling-3.0, and we have received a lot of real usage and feedback from the community. The common feedback is: they hope to have a more open and more malleable Base Model, so that they can conduct post-training based on their own data, tasks, and goals.

In the Hugging Face Community for Ling-3.0-tiny, one developer directly asked: “Can we get a base model?”

In his view, Tiny's model scale and sparse architecture are attractive to researchers with limited budgets, making it suitable for reinforcement learning rollout and scenario-specific post-training; compared with models that have already completed post-training, the community needs more of a Base Model that can continue to be shaped.

Today, we officially respond to this expectation.

01 Not only the final weights, but also the key nodes in the training process

Today, we are officially open-sourcing Ling-3.0-tiny-base and Ling-3.0-flash-base. In addition to the final Base checkpoints, we will also simultaneously release the pre-training and mid-training weight checkpoints of the two models, totaling 6 checkpoints. The community can observe how model capabilities change across training stages, compare the gains brought by mid-training and model merging, and also choose a training starting point that better suits their own data and research goals.

  • Pre-training checkpoints: Large-scale pre-training has been completed, but mid-training, WSM merging, and post-training have not yet been performed;
  • Mid-training checkpoints: Mid-training has been completed, but WSM merging and post-training have not yet been performed;
  • WSM merged checkpoints: WSM merging has been completed based on the mid-training checkpoints, but post-training has not yet been performed.

These checkpoints are suitable for continued pre-training, domain supervised fine-tuning, preference optimization, reinforcement learning post-training, distillation, and research on long context and MoE systems.

It is worth noting that after mid-training ends, we replacethe traditional learning rate decay with checkpoints weighted merging, namely WSM1 (Warmup-Stable and Merge). Since there is no learning rate decay, our Base Model is better suited forcontinued pre-training and dynamic data expansion, and supports offline exploration of different learning rate “decay curves” after training.

At the same time, Ling-3.0-tiny-base and Ling-3.0-flash-base adopt a unified training scheme, providing a clear path for strategy validation and scale expansion: first conduct low-cost experiments on Ling-3.0-tiny-base, then extend proven effective methods to Ling-3.0-flash-base.

It should be specially noted that the Base Model is not a chat model that has completed instruction alignment. We do not recommend deploying it directly as a chat service for end users without post-training, nor do we recommend using it in safety-critical applications without additional alignment and evaluation. Any production use should complete task-specific post-training, evaluation, and validation.

02 Two series of Base Models, Tiny & Flash, for different explorations in the real world

Ling-3.0-tiny-base: a code and post-training foundation better suited for researchers to get started with

Ling-3.0-tiny-base (7.9B total parameters, 1.3B activated parameters), with its relatively compact total parameters and activated parameters, provides more researchers with a solid open starting point that can continue to be trained and shaped.

Based on the model capability comparison in the table below, the model achieves higher results in most evaluations with about 50% of the total parameter count of the previous generation; in comparisons with models of the same scale, it demonstrates more competitive code capabilities.

This makes tiny-base especially suitable for:

  • conducting continued pre-training, supervised fine-tuning, and post-training in the code domain;
  • conducting budget-sensitive reinforcement learning rollout and post-training research;
  • use in university teaching, model behavior research, and MoE ablation experiments;
  • validating domain adaptation routes for specific industries and tasks.

Ling-3.0-flash-base: an open foundation balancing code, reasoning, and long-context capabilities

Ling-3.0-flash-base (124B total parameters, 5.1B activated parameters) has more ample parameter capacity and a sparse activation design, providing researchers and development teams with an open foundation that can continue to be trained, expanded, and adapted to real scenarios.

In the evaluations in the table below, the model performs prominently in code, complex reasoning, and long-context directions. Compared with base models whose total parameter count is about 2–3 times larger, the model's overall performance still has advantages.

This makes flash-base suitable for continued training and exploration in the following directions:

  • code models for software R&D and large codebases;
  • professional models requiring multi-step analysis and complex reasoning;
  • Agent foundation models for long documents, long workflows, and long-horizon tasks;
  • continued pre-training and post-training in professional fields such as finance, healthcare, scientific research, and industry.

Benchmark describes Base Model capabilities under specific settings, and is not equivalent to chat experience, Agent execution results, or production system performance. Tiny and Flash also each have capability boundaries in some evaluations such as knowledge and mathematics, and we hope the community uses this as a reference for choosing a training foundation.

03 Why open-source Base Models?

The real world does not have only one kind of task, nor only one correct answer.

Real-world tasks often involve long-term goals, professional knowledge, complex environments, tool use, and responsibility boundaries. Healthcare, finance, scientific research, industry, and personal assistants require different data, feedback mechanisms, and safety constraints. An official post-trained model can provide general capabilities, but it cannot replace the training, alignment, and validation that each domain conducts based on real tasks.

The Base Model retains more original and malleable pre-training capabilities. Open-sourcing the Base Model enables researchers and developers in the community to independently design continued pre-training, supervised fine-tuning, preference optimization, reinforcement learning post-training, and distillation routes according to their own data, tasks, and value preferences. This is an important prerequisite for open intelligence to enter the real world.

We believe that open-sourcing foundation models is true open source.

Open-sourcing post-trained models is handing capabilities that have already been formed to the community; open-sourcing Base Models is enabling the community to start from the foundation and continue training and shaping intelligence for real scenarios.

By open-sourcing the Base Models of Tiny and Flash, as well as the key checkpoints in the model training process, we hope that more researchers and developers can start from the pre-training foundation and use their own data, tasks, and feedback to let models continue to grow in the real world.

04 Start here

Ling-3.0-tiny

🤗 Hugging Face

🤖 ModelScope

Ling-3.0-flash

🤗 Hugging Face

🤖 ModelScope

Note: 1. WSM technical paper: WSM: Decay-Free Learning Rate Schedule via Checkpoint Merging for LLM Pre-training (https://arxiv.org/abs/2507.17634） 

Model release technical sharingPrevious: Ling-3.0-flash-Fin: an open model for real-world financial workflowsNext: From running to training: Ling-3.0-tiny × ASystem AReno, closing the single-machine Agentic RL loop