Back to listFrom running to training: Ling-3.0-tiny × ASystem AReno, closing the single-machine Agentic RL loop — August 14, 2026 · 14 min read

The fact that a model can run locally does not mean it is naturally suited to a specific business.

Once it enters a real workflow, the model faces more than just "how to answer": it must understand business rules, respect process boundaries, call tools correctly, and produce results that downstream systems can continue to process. A customer service Agent may pick the right refund tool but fill in the wrong key parameters; a process Agent may understand the user's intent but miss an approval step; structured extraction may look complete, yet its fields cannot be parsed by the system; a code Agent may understand the requirement but fail to modify the specified file according to the constraints.

Prompts can supplement the requirements of the current task, retrieval can supply missing knowledge, and tools can extend the model's capacity for action. But when the same type of behavioral error keeps recurring in repeated workflows, what the business needs to solve is not just "reminding the model one more time," but enabling the model to learn rules, tool usage, and action boundaries more reliably.

This is exactly where post-training can add value. Especially in local business scenarios, enterprise internal data, failure trajectories, and feedback signals are often constrained by data boundaries; business rules, tool interfaces, and process states also keep changing. The model needs to complete, in an environment close to the business frontline: baseline evaluation → feedback definition → post-training → re-evaluation in the same environment, thereby forming a sustainable, iteratively improving task-adaptation loop.

01 For real tasks, Tiny x AReno offers a lightweight post-training practice path

图

To address the problem that models still need further adaptation after entering local business, Ling-3.0-tiny × ASystem AReno offers a more lightweight post-training practice path: start from one machine, a small model, and a well-defined task; first observe the model's behavioral boundaries in the task, then turn rules and results into feedback, update the model through local post-training, and finally return to the same environment for re-evaluation.

Among these, Ling-3.0-tiny provides a model foundation that can run and iterate locally; AReno is responsible for organizing task feedback into a trainable, re-evaluable post-training loop. The AReno (ASystem Reinforcement Learning Nano) supporting this path is a localized large-model post-training toolkit initiated by the Ant ASystem team and jointly maintained by the open-source community, supporting RL, SFT/DPO-style training, local model serving, and Agentic RL. It integrates the training framework, inference service, and low-level operators into a single-node workflow, so developers can run training, serving, and task re-evaluation starting from a base model without first building a cluster or separately stitching together multiple systems.

To make the whole process clear and reproducible, this collaboration chose tic-tac-toe as the minimal validation task, performing Agentic RL post-training on Ling-3.0-tiny via AReno on DGX Spark. It should be made clear that tic-tac-toe is not the final business problem to be solved, nor is it a real-world case of Bailing; it is a minimal experimental environment with clear rules, explicit feedback, and verifiable results.

02 A minimal validation task: running the Agentic RL loop with tic-tac-toe

Tic-tac-toe has only 9 squares, with clear rules and easy win/loss determination, making it well suited to validating a post-training loop. Yet it is not a simple single-step classification task: the model needs to understand the current board state, execute actions through the choose_square tool, select legal positions, avoid repeated moves, and complete a winning move when the opportunity arises.

This minimal task simultaneously covers state understanding, tool calling, environment judgment, feedback design, and re-evaluation in the same environment, and is therefore sufficient to validate an Agentic RL loop.

The complete experimental path consists of four steps:

  • Baseline evaluation: first let Ling-3.0-tiny play directly in the same environment, observing whether it can understand the state, call tools correctly, and select legal actions;
  • Define feedback: turn task rules, illegal moves, optimal moves, and winning moves into clear Rewards;
  • Post-training adaptation: use AReno to organize rollout, Reward, training, and sampling into a loop, so the model updates its policy through interactive feedback;
  • Re-evaluation and validation: return to the same environment to play again, and judge whether the model has truly improved using visual performance, the Reward curve, and response_len together.

Before training: Tiny can already understand the rules, but its actions are not yet stable

Before training, Ling-3.0-tiny was already able to understand part of the task context and the rules of tic-tac-toe, but once in the interactive environment, its actions were not yet stable enough. For example, in its initial performance, the model made judgment errors and even placed moves on already occupied squares.

This type of problem is typical in Agent tasks: the model appears to "know the rules," but once it enters a real interactive environment, tool-call parameters, state understanding, and action selection all expose problems.

图

The interactive environment can further break the problem down: whether each step is legal, whether it violates state constraints, and whether it misses a better choice can all be directly evaluated by the environment. Compared with looking only at offline Q&A, developers can more easily determine whether the model is erring in rule understanding, state understanding, tool calling, or action strategy.

Dataset design: letting the model face reproducible task states

To make the training process reproducible, this experiment first generated a batch of intermediate tic-tac-toe positions, so the model learns how to choose the next move under different board states.

Data generation follows these rules:

  1. Create an empty 3×3 board, with empty squares represented by . ;
  2. X moves first;
  3. Make 0–6 random moves;
  4. Each move can only be randomly chosen from the currently legal empty squares;
  5. Once a win/loss occurs or the board is full, stop placing further moves;
  6. Finally keep only positions where three conditions hold simultaneously: it is X's turn, the game is not over, and the position is not duplicated;
  7. Use a fixed random seed, so the same parameters can repeatedly generate the same data;
  8. If the random process ends on O's turn, the generator additionally places a random O so that it is ultimately X's turn.

An example data item is as follows. The model needs to choose the next move based on the current board state:

{"id":"generated-00000","board":[["X",".","O"],[".",".","."],["X","O","."]]}

Reward design: turning task rules into training feedback

To make the training signal as direct as possible, this practice designed a concise Reward for the tic-tac-toe task, clearly expressing the most critical behavioral constraints:

  • No legal choose_square tool call: -1.0;
  • Parameters cannot be parsed, or an occupied square is chosen: -1.0;
  • The current move makes X win immediately: 1.0;
  • No immediate win, but it is an optimal move: 0.8;
  • The move is legal but not optimal: 0.0.

This Reward constrains three things in sequence: first, the model must learn to act using the correct tool format; second, the model must respect the environment state and cannot choose a square that is already occupied; finally, on top of being "legal," the model needs to further pursue better strategies.

At this point, the rules and outcomes in tic-tac-toe have been transformed into feedback signals that can enter the post-training loop.

DGX Spark hands-on: the model doesn't just run — it also becomes more stable through feedback

With the baseline problem and feedback rules clearly defined, the AReno side post-trained Ling-3.0-tiny in the DGX Spark environment, using the GSPO algorithm for training 400 steps。

During training, rollout/rewards_mean rose from an average of about **-0.5** to around 0.4 Combined with the task definition and subsequent re-testing, this means the model is gradually reducing illegal moves, increasing the proportion of valid actions, and learning to choose more reasonable move strategies.

图

At the same time, response_len dropped to about 850 tokensThis shows that the model became more concise when completing the task, reducing unnecessary lengthy reasoning and ineffective output. For Agentic tasks, the model not only needs to do things right, but also needs to complete tasks in a more stable and efficient way.

图

One metric reflects that task feedback is improving, while the other reflects that model output is converging. Combining the two gives a more complete view of how post-training changes the model's behavior in a specific task.

But metrics are not the end point. A more intuitive validation is to put the trained model back into the same interactive environment and observe again how it calls tools, selects actions, and respects the board state.

After training: Tiny's stability improves and its choices become more reasonable

图

Re-testing in the same environment shows that the trained Ling-3.0-tiny is much more stable in tool calling and action selection, can better respect the board state, and can make more reasonable choices at key positions.

A comparison in the same environment before and after training shows that the model's tool calling and action selection in this task tend to be stable.

It should be emphasized here: this result shows thatin the tic-tac-toe experimental environment, where the rules are clear and feedback is verifiable,post-training is changing the model's task behavior; it is not equivalent to an overall improvement in the model's general capabilities. The value of this demo lies in validating a task-adaptation path that can be observed, reproduced, and transferred.

03 What kinds of business tasks are more suitable for post-training?

Not every task requires post-training. If a single prompt adjustment can stably solve the problem, or if the task mainly lacks external knowledge, lighter-weight context, retrieval, or tool solutions can be chosen first.

When the following conditions appear at the same time, the value of post-training becomes clearer:

  • The task rules are relatively clear. What counts as correct, incorrect, legal, violating, or better behavior can be clearly described;

  • Model errors can be reproduced. The same type of error keeps appearing in similar tasks, rather than being a one-off sporadic problem;

  • Results can be evaluated. The environment, rule programs, or human feedback can determine whether the model has completed the task, and form supervision data, preference signals, or Reward;

  • The task occurs repeatedly. The model needs to maintain stable behavior across a batch of similar processes, not just complete a single conversation;

  • The goal is to change model behavior. Developers want the model to develop more stable formats, tool usage, decision preferences, or process constraints over the long term.

For example:

  • Tool-call repair: Reduce errors in tool names, parameter formats, and call timing;

  • Structured extraction: Make the model consistently follow fields, Schema, and output formats;

  • Domain instruction following: Complete tasks within professional rules and business boundaries;

  • Personal writing format adaptation: Stably learn specific expressions, structures, and layout preferences;

  • Business process Agent: Correctly call tools, fill out forms, advance processes, and handle environment feedback in multi-step tasks.

04 Why choose Ling-3.0-tiny?

The core reason for choosing Tiny is not that it is "small enough," but that while lowering the threshold for local running and experimentation, it still has the capability foundation to enter reasoning, tool calling, and Agent tasks.

Ling-3.0-tiny has 7.9B total parametersbut activates only 1.3B parametersper Token during inference, and is a natively hybrid reasoning model; the three open-source versions BF16, FP8, and INT4 also let developers flexibly choose according to hardware environment, capability needs, and deployment cost.

In"Ling-3.0-tiny officially open-sourced: bringing 1.3B activated parameters into real tasks" Tiny has already demonstrated local running capability on devices such as DGX Spark, MacBook, and Mac mini, and can also enter real workflows such as local knowledge bases, code assistants, UI automation, and enterprise task agents.

For post-training, model size directly affects whether developers can quickly complete a task-adaptation loop. If the model is too large, one experiment often requires higher VRAM, more complex parallel strategies, and longer waiting times; after Reward, data, or Agent logic changes, it is also difficult to quickly enter the next round of training and re-testing.

The value of Ling-3.0-tiny lies in shortening this feedback chain:

  • Lightweight and trainable. A model scale of 7.9B total parameters and 1.3B activated parameters is more suitable for completing training, debugging, and reproduction experiments in lightweight environments such as DGX Spark;

  • Fast iteration. Adjustments to Reward, data, Prompt, and Agent logic can more quickly enter the next round of training and re-testing;

  • A foundation for exploring real tasks and Agentic RL. Tiny can already enter real tasks, tool calling, and Agent workflows, and can continue to adjust action strategies through environment interaction and rule feedback.

05 Why choose AReno?

The core reason is: what AReno hopes to lower is the engineering threshold from "having a task-adaptation idea" to "actually completing training and re-testing."

In the past, reinforcement learning post-training often meant a whole complex system including training frameworks, inference services, operator dependencies, environment configuration, cluster resources, Reward design, and Agent trajectory collection. If any link was not connected, an originally very specific task-adaptation idea could stall at "not yet running."

AReno hopes to shorten this path. Currently, it supports:

  • SFT / DPO-style training;
  • RL post-training;
  • Agentic RL；
  • Local model serving;
  • Custom Reward Functions;
  • A closed loop of training, inference, sampling, and optimization.

AReno connects local serving, rollout, Reward, and training into a closed loop, and allows developers to replace data, environments, and Agent Functions; developers can therefore get tasks running locally, see the changes, and migrate them to their own workflows.

06 How to reproduce the experiment and migrate it to your own business?

AReno already provides a complete tutorial for running training on DGX Spark via Docker. Developers can directly follow the guide to reproduce the tic-tac-toe experiment:

DGX Spark Tic-Tac-Toe Post-Training Tutorial（https://github.com/inclusionAI/AReno/blob/main/examples/agentic/tictactoe/DGX_SPARK_GUIDE_CN.md） 

Once it runs successfully, you can replace the dataset, Reward, or Agent Function to turn this minimal experiment into a post-training experiment for your own task; the same method can be migrated to the aforementioned tasks such as tool calling, structured extraction, domain instructions, and business-process Agents.

What is truly worth reusing is not tic-tac-toe itself, but the method behind it:

find a reproducible error → define verifiable feedback → complete local training → return to the same environment for re-testing.

This is also the core asset that this Tiny × AReno practice hopes to deliver.

Next, AReno will continue to add more cases oriented toward small models, Agentic RL, developer practice, and university teaching, helping developers start from smaller, more intuitive tasks and gradually migrate post-training methods to their own real-world scenarios.

07 You are welcome to download and use Tiny, and you are also welcome to contribute to AReno**

Ling-3.0-tiny is now officially open source, with BF16, FP8, and INT4 versions available. You are welcome to download and use it according to your device and task needs:

🤗 Hugging Face

https://huggingface.co/inclusionAI/Ling-3.0-tiny 

https://huggingface.co/inclusionAI/Ling-3.0-tiny-fp8 

https://huggingface.co/inclusionAI/Ling-3.0-tiny-int4 

🤖 ModelScope Community https://modelscope.cn/models/inclusionAI/Ling-3.0-tiny 

https://modelscope.cn/models/inclusionAI/Ling-3.0-tiny-fp8 

https://modelscope.cn/models/inclusionAI/Ling-3.0-tiny-int4 

You are also welcome to visit the AReno GitHub repository (https://github.com/inclusionAI/AReno），复现这次  Spark post-training experiments, submit Issues, join discussions, or contribute Tiny personalized training Cases.

If you are interested in the following directions, you are welcome to join the AReno open-source community:

  • Lightweight model post-training;
  • Agentic RL；
  • Reward Function design;
  • Tool calling and workflow Agents;
  • Small-model task adaptation;
  • Localized AI toolchains;
  • AI teaching and developer communities.

08 From local running to better matching real business

A model entering the local environment is only the first step; whether it can continuously adapt based on business feedback is what determines whether it can truly enter the workflow.

What Ling-3.0-tiny × ASystem AReno validates this time is not an already completed industry solution, but a lightweight, transparent, and reproducible task-adaptation path: from getting it running, to training it, and then returning to the task to verify whether it has truly improved.

What we hope to open up is not only model weights and training tools, but also a method that can be verified, criticized, modified, and further created.

Open intelligence for the real world.

Let intelligence understand tasks, call tools, execute actions, and accept feedback in real environments; let results be verifiable, let boundaries remain clear, and let responsibility always be borne by humans.

Technical Sharing Reinforcement LearningPrevious postLing-3.0 tiny & flash Base Models Officially Open-SourcedNext postLing-3.0-tiny Officially Open-Sourced: Bringing 1.3B Activated Parameters into Real Tasks