Diffusion Intelligence releases Clover-35B-A3B-Preview, a 35-billion-parameter diffusion language model

Who would have thought that the "diffusion model," which we associate with generating images and videos, could also write code, generate web pages, call tools, and act as an Agent!

Just now, Diffusion Intelligence released a brand-new diffusion language model—

Clover-35B-A3B-Preview.

Although the name has the extra word "diffusion," in actual use, whether it's writing code, operating web pages, calling tools, or generating HTML, it works much like the large language models we are familiar with.

My first impression from using it personally can be summed up in one word: "fast."

For example, when asked to generate a solar system simulator, Clover thought for about 1 second, and then in just 4-5 seconds, an interactive solar system simulator was produced.

For another example, I threw a pile of resume information at it and asked it to organize it into LaTeX resume code; it took just over ten seconds—done~

To be honest, anyone who has used GPT should know what it feels like to stare at "thinking" for ages; otherwise, there wouldn't recently be all this fuss about Jev, which focuses on rapid decision-making~

And speaking of Jev, Clover really did compare speeds with it this time.

In the 49 tasks of the web Agent benchmark WindTunnel, with 3 attempts per task, Clover's median execution time was 1.52 seconds, while the Jev + Mercury 2.5 combination took 3.21 seconds—less than half the time.

The attempt success rate was also slightly higher, at 98.6% and 95.9%, respectively.

Compared with Gemini in the same table, it is several times faster!

All right, all right—you diffusion model, are you trying to drive autoregressive models crazy? (doge)

Back to the model: Clover-35B-A3B-Preview this time adopts an MoE architecture, with 35 billion total parameters, about 3 billion parameters activated per token, and support for thinking mode.

It further advances the team's diffusion language model from the 7 billion parameters of the previous Dream 7B to a total parameter scale of 35 billion.

More critically, after the parameter count increased, Clover's speed did not slow down because of it; in comparisons of the same size, it surpasses autoregressive models.

In addition to the interactive pages and code generation above, in the official demo, Clover further demonstrated Agent capabilities such as complex tasks and WebUI generation.

Through WebMCP, Clover can continuously handle six tasks, involving invoicing, course registration, amount aggregation, event status queries, as well as customer information queries and invoice information queries.

From the comparison video below, we can see that Clover not only successfully completed the tasks but also completely left Gemini 3.6 Flash behind in execution speed.

In an online shopping scenario, Clover can also, as required, find a T-shirt in the specified style, black, size L, add it to the cart, fill in guest information, and then complete the test checkout process.

Honestly, this kind of continuous task handling is actually what is worth paying attention to in this demo.

You should know that although diffusion language models have the speed potential of parallel generation, their past performance in Agent tasks has actually not been stable enough:

The model's tool-call format may be wrong, and it is also prone to repeatedly trying the same step without adjusting actions based on the returned results.

Clover's progress this time is reflected in these continuous operations: calling tools, reading feedback, and continuing execution until the task is completed.

Specifically for benchmarks, according to the data published by the team, Clover scored 65.3 and 67.9 on the tool-use-related τ³ Overall and BFCL-v4 evaluations, respectively, ranking among the top.

Having looked at Agent tasks, let's now look at code.

I also asked Clover to generate a personal Agent web interface. It looked quite decent, and the whole generation process took just over ten seconds.

On the code evaluation side, Clover also delivered a good set of results.

On SWE-bench Verified, which tests the ability to solve problems in real code repositories, it scored 73.6; on Terminal-Bench 2.1, which tests terminal task execution ability, it scored 51.7.

It can be said that from generating interactive pages, to calling tools and running through business processes, to code evaluation, Clover, as a diffusion language model, has indeed taken on a lot of Agent work this time.

Hey, this is interesting. The diffusion approach that we associate with denoising and image generation—how exactly does it generate text and write code in a language model?

Could it be that the demos above were really produced by AIGC?

Generally speaking, the autoregressive (AR) language models we are familiar with predict the next token based on the preceding text, generate one, then generate the next—what people often call "word chaining."

The token here can simply be understood as a text fragment processed by the model.

Compared with the one-by-one serial prediction and generation of autoregressive models, diffusion language models use a different generation method: allowing multiple positions to be updated simultaneously, and then, after several rounds of refinement, gradually arriving at the result.

The diffusion denoising here can be understood as gradually restoring masked or perturbed content into text that fits the context.

Specifically for Clover, it adopts block diffusion: dividing the content to be generated into several blocks, with blocks advancing in sequence, and multiple positions within a block updated in parallel and referencing one another.

In this way, a single round of computation has the chance to advance the generation of multiple positions, reducing the waiting caused by generating one by one.

But the problem is that being able to generate together does not mean it can be written well in one round.

Because diffusion models often require multiple rounds of denoising to gradually settle the result. With too few rounds, the content may not yet be properly adjusted; with more rounds, the computational overhead also increases.

When it comes to code and Agent tasks, the context also keeps growing longer. Previous requirements, operations already executed, and results returned by tools may all affect the next step, and saving and reading historical information also becomes an efficiency issue.

Clover's improvement approach is to let historical memory and current generation each play their own role.

It restructured the Gated Delta Network (GDN) around block diffusion: the already-generated context is compressed into a continuously updated state for subsequent steps to read; the currently generated block, through bidirectional attention, lets the tokens within the block reference one another.

At the same time, Clover treats the readout of historical memory as an independent information unit, participating in attention computation together with the current tokens, letting the model learn on its own how to weigh prior text against current content; then, through learnable block size calibration, it reduces the problem of historical information being diluted as blocks grow larger.

In other words, the goal of these adjustments is to better coordinate parallel generation, historical information utilization, and computational efficiency—and this is also the key to Clover pushing diffusion language models further into code and Agent tasks.

And apart from Clover, the reason diffusion language models have attracted attention over the past two years mainly lies in their two major advantages over autoregression:

On the one hand, diffusion language models allow multiple positions to be generated simultaneously, offering the chance to reduce the serial waiting caused by token-by-token generation. For Agents that need to repeatedly generate code and call tools, this potential is especially noteworthy.

On the other hand, during generation, preceding and following content can reference each other. Under the design of bidirectional modeling, multiple positions can be jointly adjusted through iterations, providing another way to handle dependencies among content. The interaction within Clover's current block is one such implementation.

Around these possibilities, academia and industry have already offered different attempts.

ELF, which He Kaiming participated in this year, attempts to generate language in a continuous embedding space; Google's DiffusionGemma explores parallel text generation; major domestic companies are also researching diffusion language models for code tasks.

Specific methods differ, but everyone is trying to turn another generation approach into usable capability and efficiency.

This time, Clover has put its own progress into code, tool calling, and continuous tasks, leaving it to users to try out and verify.

Finally, let us return to the team behind Clover—

Diffusion Intelligence.

It is reported that Diffusion Intelligence was jointly founded by Professor Kong Lingpeng of the University of Hong Kong and his doctoral students Gong Shansan and Ye Jiacheng. The team members' exploration of diffusion language models can be traced back to 2022.

That year, they proposed DiffuSeq, studying how to use diffusion models to accomplish sequence-to-sequence text generation: given a piece of text, generate another piece of text according to conditions.

At that time, one hurdle to overcome was applying diffusion methods, which excel at handling continuous signals such as images, to discrete natural language.

By April 2025, core team members participated in launching Dream 7B, advancing the research to a 7-billion-parameter language model.

According to the results announced at the time, Dream 7B achieved performance comparable to autoregressive models of the same scale on general, mathematical, and code tasks, and also demonstrated planning ability and flexibility in generation order.

Then came this time's Clover, where the model scale continues to expand, and writing code, calling tools, and continuously executing tasks have all become key showcases.

This route has thus gone step by step from initially studying how to generate text to how to do complex tasks correctly and quickly. Now, the team is still extending toward multimodality and on-device deployment.

According to the introduction, Clover will also launch a multimodal version in the future; the team is advancing the combination of diffusion language models with multimodal understanding and action generation, and continuously carrying out adaptation and validation for on-device scenarios. The collaboration with Acrab is advancing system adaptation and software-hardware co-verification for on-device scenarios.

According to the team's plan, they will continue to explore larger-scale models, improve the stability of long-horizon tasks, and advance on-device applications through inference optimization, quantization, and deployment adaptation.

Last but not least, let us use this newly released Clover to open up a thought experiment:

When humans think, do they really push forward one sentence after another, or do they first have a vague whole and then gradually fill in the details—

Is the brain more like autoregression, or diffusion?