Hands-on test of Kimi K3.1 Pro: Terminal-Bench 4.0 rises to 67.2%, and it can fix simulator errors on its own

By Xiang Xianzhi

With K3.1 Pro, my first reaction was: it's fast. Even though I was using Pro, it was much faster than the previous K3.

When K3 first came out, its speed was genuinely hard to rate. During this test, everything felt silky smooth. It certainly still can't match the feel of the various flash models, but the thinking speed is entirely normal-to-fast.

A bunch of people were previously shouting that domestic large models would stop making Pro versions after distillation was banned, and Kimi has come right out to slap the haters in the face.

Speed and thinking ability are combining again

This time I didn't turn on high-speed mode; the speed of regular Pro alone was enough to satisfy me. Kimi also launched Pro Highspeed, which per the official description delivers the same results as regular Pro with model output speed about 2–3 times faster.

But that's not the speedup from K3 to K3.1. Why regular Pro got faster—officials didn't separately explain what adjustments were made on the inference side. The technical blog talks more about training: Kimi increased post-training investment, having the model undergo joint reinforcement learning across hundreds of downstream domains, and also improved long-context handling and the environment for executing tasks.

On the leaderboards, task-execution scores rose the most noticeably. In the composite index published by independent evaluator Artificial Analysis, K3.1 Pro rose from K3's 44 points to 55 points, ranking 4th. On the terminal task test Terminal-Bench 4.0, it rose from 12.6% to 67.2%, placing first among the results released that time. It requires the model to complete tasks in a computer terminal, with success only counted if it passes all checks.

In independent evaluations, task execution improved markedly while long-context reasoning dipped slightly

For users without a technical background, the two office-related scores are more useful references. GDPval-AA compares the quality of documents, spreadsheets and presentations the model produces, and K3.1 Pro ranks 3rd. AutomationBench-AA tests the ability to complete work across email, spreadsheets and customer management systems, and Pro's target completion score rose from 58.3% to 73.3%.

In the results table Kimi published, Pro also took two leads: 51.7% on the long-task coding test SWE-Marathon v1.1, higher than Opus 5.5's 46.3%; and 64.4% on the financial document Q&A OfficeQA Pro v2, higher than Opus's 62.2%. Both rank first among the models listed in that table. Here Pro's scores are self-tested by Kimi, while opponents' scores partly come from external leaderboards.

Progress is uneven. In the official full HLE hard-problem test, Pro with tools scored 70.0%, higher than Opus 5.5; without tools, Pro scored only 51.1%, lower than Opus's 59.8%. In AA's long-context reasoning test, K3 (Max) had 88.7%, while Pro had 86.3%.

The official humanoid walking demo is more intuitive: a virtual human keeps falling at first, and after several rounds of training gradually learns to walk continuously. K3.1 Pro wrote the simulator, arranged training and evaluation, then revised its methods based on experimental results. Throughout, the arm movements were never ideal, and adjusting rewards didn't fix it; the model then inspected the simulator, fixed 36 errors in shoulder muscle moment-arm calculations, and retrained the controller.

In the official demo, the virtual human goes from falling to learning to walk continuously

The human body's movements are generated by simulation and control policies, and Kimi is responsible for continuing to improve training based on the results. It didn't stop at adjusting rewards—it also found errors in the simulator, and that part interests me more than how human-like the final walk is.

Doing this kind of long task naturally increases call volume. This time Kimi lowered the API output price for regular Pro: per million output tokens, K3 is 100 yuan, Pro is 68 yuan, and Highspeed costs 136 yuan. A token is the unit platforms use to calculate text and code usage; monthly subscribers still use their plan quotas.

The high-speed version of Kimi Code also consumes about twice the quota, and the speedup only happens in the model output stage. Reading and writing files, executing commands and running programs still take the original time. When regular Pro is already fast enough, I'll save my quota for later revisions.

Speed is up, but the quota may not be enough

Some users feel regular Pro is fast enough. One call record he posted reached 111 tokens per second, saying it's sufficient even without the high-speed version.

Chetaslua, who received early official access from Kimi, posted a web demo, saying it was generated by the model in one pass, and included a CodePen link so readers can open it and see the result directly.

Other users ran into quota problems during back-and-forth revisions.

LanceYang gave the model five reference images and asked it to make an interactive 3D game controller. After the first version he revised it twice more, but the part shapes and spatial relationships were still wrong, and the model hit the usage limit again while verifying the revision results. He was on the 199-yuan plan and had already used part of his quota before starting on the controller.

In the well-known pelican-riding-a-bicycle test, two users in another thread reported very different consumption. One user on the 199-yuan plan chose the high-speed version and the highest thinking tier, and after generation asked for "continue optimizing," which nearly used up the rolling quota. Sanyeg used the 599-yuan plan with the same model tier, but said a single pelican task consumed only about 6%. The plans, tasks, and revision processes were all different, so it is hard to judge which is more cost-effective from the percentages alone. So overall, token consumption now seems a bit puzzling.

Under Kimi's rules, all features share the monthly quota; the model, context, and tool calls all affect consumption, and Code additionally has a rolling usage limit. Whether a user can successfully complete a piece of work depends on the total usage from generation through revision.

The scenarios have small bugs, but it can fix them itself

I chose seven programming problems, started a new session for each, allowed Kimi to run, check, and revise on its own, and then reviewed the first delivery afterward. We did not change the code for it. Images, GIFs, and prompt excerpts are below; the full prompts and input materials are included in the release package.

"Pelican riding a bicycle" has been tested too many times, and we worried about benchmark optimization. So we chose: make the bicycle ride the pelican.

Actual prompt (excerpt): Two equally sized stages play simultaneously on the left and right of the frame; the left shows "a pelican riding a bicycle," and the right shows "a bicycle riding a pelican." ... The bicycle is supported by the pelican's back, both wheels are off the ground, and the pelican strides forward with alternating legs.

The bicycle got above the pelican but did not land on the bird's back

Kimi made a left-right comparison: on the left the pelican pedals, on the right the pelican walks, and the bicycle hovers above it. The positions were swapped, but a gap remained between the bicycle and the bird's back, so it was not clear that the pelican was supporting the bicycle. This part should logically connect the pelican and the bicycle at the front node, which caused the later floating; another small problem is that the bicycle wheels kept spinning, which is a bit illogical, but it can also be optimized.

The physics problem performed better. We set up small balls colliding inside a rotating heptagon and observed the actual collision and motion behavior; the result was quite good.

Actual prompt (excerpt): A regular heptagon with circumradius 320 rotates clockwise at 0.2 radians/second. Inside are 20 small balls of radius 12, numbered 1 to 20, initially non-overlapping, subject to downward gravity of 500 pixels/second²; ... wall bounces must account for the local velocity of the rotating wall.

Small balls rotate with the frame, get carried up, then roll back to the bottom

By continuously running the simulation and checking ball positions, I found no wall penetration or lost balls. It also factored the wall's motion velocity into the bounce process. The first draft delivered for this problem is already usable to demonstrate gravity and collisions.

For the sound analysis problem, I additionally swapped in low- and high-pitch files to check whether the spectrum truly reflected the input.

Actual prompt (excerpt): Analyze the actually playing sound via the Web Audio API and display the spectrum, RMS loudness, and dominant frequency in Hz; the frequency axis must cover at least 0 to 4000 Hz, use interpretable real data, and pre-programmed rhythmic animation is prohibited. The volume slider affects the output and analysis results. ... An included F05-demo.wav can be used for development, but the program must support other audio in the same format.

With new audio swapped in, the spectrum and readings still correspond

After switching to low- and high-pitch files, the chart and frequency readings matched. When the volume was halved, the measured amplitude was also nearly halved; after muting, the amplitude went to zero and the dominant frequency value was no longer displayed.

However, short sounds linger on the chart for a while before disappearing. It is already sufficient for seeing pitch and loudness, but to make the visuals keep up with the beat, more adjustment is needed.

The pixel runner is only 12×8 pixels and must use four frames to show running.

Actual prompt (excerpt): Use at most 32 fixed RGB colors to draw a rainbow road, starry sky, and a 12×8-pixel runner. The runner has a recognizable four-frame running loop, and the body always maintains a 12×8 logical size; the road scrolls left, and the character stays at the horizontal center of the frame.

Four poses combine into a running motion, and the character still keeps a pixel style

The size, color count, and number of animation frames all met the requirements. The little figure's arms and legs swing alternately, the rainbow road scrolls backward, and although the image is simple, the running motion is recognizable.

There was also one task asking the same card to morph continuously among controls such as a button and a player.

Actual prompt (excerpt): The same card shell at the center of the frame transforms in sequence within 16 seconds into a button, loader, success check, player, volume control, switch, bar chart, and notification bar, then back to a button. ... Use continuous changes in shape, corner radius, size, icon, and text to express the states; do not merely switch whole pages.

The same card morphs among a button, player, and other controls

Most of the changes were smooth; the problem was at the end: the notification text disappeared first, leaving a blank button on screen, and when the next cycle began, the text on the button suddenly appeared. It is easy to miss in screenshots, but continuous playback reveals it.

When returning to the button, the text suddenly appears

In the part testing 3D modeling ability, we chose the Along the River During the Qingming Festival case and specifically required the people and boats to move. When crossing the bridge, foot height must change with the bridge surface; porters must first pick up goods, then walk to the dock and set them down. Only by zooming in can you see clearly whether it coordinated the actions with the scene.

Actual prompt (excerpt): Please create a 3D "Along the River During the Qingming Festival" scene that users can freely roam in a browser, focusing on the Bian River, the wooden arch Rainbow Bridge, the market at the bridgehead, the shops along the river, and the city gate in the distance. ... The characters should have recognizable bodies and limb movements, continuously performing different activities, including at least walking along the street, crossing the bridge, trading at stalls, and carrying goods at the dock; the boats should sail along the river, and the people on board should move with the boats. ... Pedestrians move along passable roads, docks, and bridge decks, with their feet attached to the current ground or bridge deck height, not walking into the river, buildings, or stalls, and not passing through the railings.

The Rainbow Bridge, street market, and dock are all made into 3D scenes that can be viewed from all angles

During the production process, I saw small figures floating off the ground, and I also saw people walking into the river and passing through shops. These problems all happen at first. Then Kimi still optimizes on its own.

Kimi began adjusting step by step on its own. For the clipping issues and the problem of characters walking across the river, we did not add extra prompts for these basic logic issues, and it also corrected the characters' height on its own and changed the walking routes that passed through shops.

The most impressive thing is that the shadow calculations are all very accurate, and you can clearly feel that the model has begun to understand the real laws of the physical world.

These modifications happened during Kimi's own checking process; we did not add modification prompts.

In the delivered version, the porters pick up goods and walk to the dock, then return empty-handed, and customers also stop in front of the stalls. Up close, problems can still be found: when pedestrians walk, their bodies pass into the shop walls. The river channel also has problems: the water surface is blocked by the ground, and the boats look like they are sliding on the ground.

The delivered version still has the problems of pedestrians passing into shops and the water surface being blocked by the ground

Finally, I asked Kimi to make an offline video editor. The timeline, preview, and export must use the same editing result, and refreshing the webpage must also retain the modifications just made.

Actual prompt (excerpt): Each clip can have its in-point and out-point set precisely; support splitting clips at the current playhead position, deleting clips, rearranging clip order, and undoing and redoing edits. ... Actually export a playable WebM video, allowing real-time export. The export must reflect the content after cutting and rearranging, and must not download the original video to pass it off as an export. ... Save project state locally; after refreshing or reopening, the imported materials and editing results can be restored.

After the cut clips are rearranged, they can be exported as a finished video

We imported the two recordings of the runner and the pelican, cut one segment from each, and adjusted them so that the runner came first and the pelican second. Checking the exported video, the order and the cut ranges matched up. After refreshing the webpage, the materials and the cut timeline were also restored. Splitting, deleting, undoing, and redoing all worked normally in this round of operations, and this first draft could already complete this simple edit.

The exported file still has a small flaw: in the Edge browser used for testing, the total video duration cannot be read when it is first opened, and it is only recognized after playing through once. The video content is correct, but the file information has not been fully processed.

In this round, physics, audio analysis, and simple video editing already have practical uses. Several visual questions also produced first drafts that can continue to be modified, and the spatial relationships and motion transitions still need to be checked.

Kimi calls Pro the first model in the K3.1 series, and Flash or Ultra has not yet been announced. If Flash is launched in the future, I care more about whether it can reduce the cost of frequent calls.

Multiple Agents working at the same time will generate more model calls. If Flash can handle simple steps at a lower price and faster speed, developers can leave complex problems to Pro.

There were previous reports that Moonshot AI began recruiting embodied intelligence talent, and employees at some robotics companies received recruitment calls from Kimi. The report also cited people involved in interviews as saying that internally there are still reservations about the Agentic Robotics route.

Google has already tried using Flash for robots. Based on Gemini 3.5 Flash and trained through specialized embodied reasoning, it created Gemini Robotics ER 2. This model is responsible for observing the environment, planning, and monitoring tasks, and then calls lower-level action models to execute.

The human walking demonstration, plus the recruitment of embodied talent, makes it hard not to believe that Kimi also wants to continue exploring in this field.

The embodied solution I am most optimistic about right now is to let a strong model participate in design and improve training, so that control policies learn actions in simulation; during actual operation, a fast model observes the environment and plans tasks, then hands them to a lower-level control model for execution. Training can be tried repeatedly, but when a robot faces a changing environment, it must respond in time. If Kimi has a low-latency, low-cost multimodal Flash in the future, it could be used in the observation and planning part.

As for Ultra, what I expect is that it fills in the ability for harder reasoning and long tasks. Pro still has a gap in difficult problem tests without tools.

Although this test did not produce a "shocking collapse," it can actually be inferred that Kimi is advancing fairly steadily toward the real physical world and overall capability optimization, since this time it is still optimizing on the K3 base.

Kimi has already handed in its paper; now open source is just waiting for Qwen4.