Ant Lingbo Open-Sources Tactile Pre-Trained Foundation Model LingBot-Touch

Recently, a team conducted an experiment in which they installed tactile sensors on all five fingers of the Sharpa Wave dexterous hand, with the task of getting it to zip up a bag.

The action model they chose was π0.5, and when relying only on cameras, the success rate reached 39%.

The team then fed it the readings from the fingertip sensors unchanged. Logically, with an extra sense, the robot should be more adept at zipping up the bag.

With the same combination of the Sharpa Wave dexterous hand and π0.5, when connected to raw tactile input for the box handover task, the success rate dropped from 57% to 48%.

Even when the dexterous hand was switched to XHand, averaged across 6 tasks, the success rate still fell from 49.71% to 48%.

So what exactly is the reason that the robot, with an extra sense, instead becomes clumsy?

The answer is that the problem lies in the readings.

This is because tactile data is not quite the same as other data. The robot's fingers are not in contact with objects most of the time, so the readings are mostly a large expanse of "0"; even once contact with an object begins, the signal is sparse and noisy.

In addition, different sensors each serve their own function: visuotactile sensing captures the deformation of the fingertip's soft gel, magnetic tactile sensing measures the magnetic field, and piezoresistive tactile sensing records resistance... the data are all different.

For example, for the task of gripping a bottle cap, the three types of sensors can give three completely different numbers. Not only does the action model fail to understand them, it also has to use them anyway... you could say they are actively getting in the way.

So how can this situation be resolved?

Ant Lingbo has offered a solution—adding a "translator" between the sensors and the action model.

(Note: the experiment above was conducted by Ant Lingbo.)

Simply put, the data read by various sensors is first translated into the kind that the action model can understand, such as where the finger touches, how much force is used, which way the force deviates, and whether there is slipping, and then it is handed to the model.

Still the same π0.5, still the same zipping task, and with this approach, the success rate directly soared to 59%, even 20 percentage points higher than using no tactile sensing at all; for the box handover task, the success rate also returned from 48% to 63%.

And this "translator" is LingBot-Touch, the tactile pretrained foundation model newly open-sourced today by Ant Lingbo.

With its support, the bottle-cap-twisting action we just mentioned becomes very smooth:

It should be noted that the place where twisting a bottle cap easily goes wrong is precisely the small area where the fingertip pad and the bottle cap are pressed together: if the grip is slightly loose, the fingertip pad will slip on the cap surface, and after the wrist turns half a circle, the cap does not move at all. And this is exactly the scene that cameras cannot see clearly.

So how exactly does LingBot-Touch, this "translator", accomplish this?

Let us first explain clearly what LingBot-Touch does.

Simply put, it is responsible for receiving signals from three types of sensors—magnetic tactile, visuotactile, and piezoresistive tactile—and then uniformly encoding them into a kind of tactile Token, from which it reads out the contact position, the magnitude of the force, and how the direction of the force changes over time.

Its position sits right in the middle: downward, it can connect to different sensors and dexterous hands; upward, it hands these Tokens to different action models.

In this way, no matter how varied the signals from different sensors are, by the time they reach the action model, they can all be understood uniformly—truly all roads lead to the same destination.

Moreover, the lightweight version of LingBot-Touch, LingBot-Touch-Tiny, has only about 5.32 million parameters, and the technical report, code, and model weights have all already been open-sourced.

So how was LingBot-Touch trained?

Overall, the Ant Lingbo team mainly took three steps.

The first step is unifying coordinates.

The first challenge the team encountered was that the number and arrangement of contact points differ greatly across sensors. For example, some dexterous hands have 116 contact points on the thumb alone, while some fabric gloves have only a small 3×3 array.

If matched rigidly by number, they definitely would not line up.

To address this, the team first mapped the contact points of all sensors into the same standardized fingertip space, and then gave each Token an "address", which consists of time, finger, and three-dimensional position.

The address enters the model through rotary position encoding (RoPE). In this way, the model no longer requires a fixed number of contact points; even if the layout changes, it can still read them.

On this basis, each of the three types of sensors has its own lightweight entry point, responsible for converting raw readings into Tokens; behind the entry points, they share the same tactile Transformer backbone.

That is to say, the differences between sensors are absorbed by their respective entry points, while the patterns of contact itself are learned together in the backbone.

The second step is making predictions in feature space.

After unifying the coordinates, the team encountered a second pitfall, namely the large expanse of "0" we mentioned at the beginning.

If the model were made to reconstruct the raw readings, it would quickly find a shortcut: as long as it learns the "no contact" background well, the loss can be reduced very low. But in this way, although the score looks good, not much truly useful contact information is learned.

Therefore, LingBot-Touch chose the JEPA route.

Specifically, during training the model randomly masks 90% of the touch points, and keeps them masked for the entire duration; then, based on the changes in the remaining 10%, the model predicts the representations of the masked parts in feature space, without needing to reconstruct the readings point by point.

With this approach, the results were immediate: trained on the same 1,000 hours of data, LingBot-Touch's error in judging the contact center was 0.90 mm, while VAE and MAE were both above 3 mm.

For another example, when using a segment of touch to find the corresponding video frame, LingBot-Touch's top-5 recall rate was 32.07%, while the other two methods were both below 6%.

The third step is to expand the data to prevent the model from rote memorization.

LingBot-Touch's pretraining data totals 4,983 hours, about 1.33 billion frames. Two-thirds of it comes from people wearing tactile gloves to do work, and the remaining one-third comes from teleoperating the Sharpa Wave dual-arm robot.

The reason for using gloves extensively is that they do not require a robot; one person can record several hours a day in an ordinary environment. Moreover, these data are only labeled with task categories such as open, close, assembly, and pouring water, and not a single frame has been labeled with force magnitude.

However, once enough data had been accumulated, the team still had to prevent the model from "slacking off."

Because there are only three sensor layouts in pretraining, the model could very likely memorize these three sets of touch point positions outright, and then be at a loss when encountering a fourth layout.

So during training, the team randomly changes the sampling density of touch points, and then slightly shifts the position of each touch point, forcing the model to learn the patterns of contact itself.

After these three steps, it was time for real-robot validation.

The tests covered two dexterous hands, among which the high-DOF Sharpa Wave performed 20 tasks and the low-DOF XHand performed 6, for a total of 26 tasks.

Four action models participated in the tests, namely Diffusion Policy, π0.5, LingBot-VLA, and T-Rex. Each model had to be tested separately under three settings: without touch, directly feeding raw signals, and connecting to LingBot-Touch.

As a result, on both hands, the group connected to LingBot-Touch had the highest average success rate for all four models.

Compared with no touch, the success rates of T-Rex, π0.5, and LingBot-VLA increased by 23.18, 21.81, and 24.81 percentage points, respectively, while the cost was only an increase of 0.13% to 0.16% in parameter count.

Take inserting a straw with one hand as an example. The difficulty of this task lies in the moment of piercing the sealing film. As the straw presses down against the transparent sealing film, the resistance grows greater and greater, and once the film breaks, the hand suddenly goes slack.

It is hard for a camera to distinguish whether the film has been pressed concave or has already been pierced, but the fingertips can. If the force is released even one step too late, the straw will shoot straight to the bottom of the cup.

Of course, the most telling experiment is the XHand group.

XHand uses a magnetic tactile sensor whose model has never appeared in the pretraining data; moreover, all the magnetic tactile data in pretraining came from gloves worn by people. When connecting to XHand, not a single parameter of LingBot-Touch was changed.

Even so, π0.5's average success rate on the 6 tasks still rose from 48.00% when directly fed raw signals to 74.17%.

Among them, for the task of separating stacked cups, the success rate more than doubled from 32.5% to 72.5%.

This task requires the dexterous hand to take away only the topmost cup; it must use one hand to steady the whole stack of cups and the other hand to pinch the rim and pull outward. If the resistance suddenly becomes smaller while the fingertip pad is still steadily against the rim, it means the cup has come loose and force must be released immediately; if the fingertip pad itself is slipping on the rim, then it should re-grasp.

These two situations feel completely different to the touch.

In other words, the experience accumulated by human hands helped the robot understand a hand it had never touched before.

In addition, the team also compared LingBot-Touch with the previous tactile foundation model FTP-1.

When judging the contact center on Sharpa Wave, FTP-1's error was 2.19 mm, while LingBot-Touch's was 0.12 mm; on XHand, the error also dropped from 5.89 mm to 0.16 mm.

It should be noted that FTP-1 was also equipped with dedicated adapters for each of these two hands, while LingBot-Touch used the same frozen encoder.

In addition, LingBot-Touch's Scaling Law has also begun to emerge.

The Tiny version began to saturate at around 4,000 hours, while with the larger-capacity Small version, performance was still rising at 5,000 hours; the success rate on real-robot tasks likewise improved steadily as pretraining data increased.

After reviewing the technical details, if we place LingBot-Touch back into Ant Lingbo's overall technical roadmap, we can see its position more clearly.

Previously, Lingbo's general-purpose brain had already laid out three capabilities: spatial perception with LingBot-Vision and LingBot-Depth 2.0, environment interaction with LingBot-Video and LingBot-World 2.0, and dexterous manipulation with LingBot-VLA 2.0 and LingBot-VA 2.0.

As can be seen, most of these capabilities address what happens before contact—for example, where an object is, where the hand should reach, and what to do next.

But once a finger touches an object, the information vision can provide diminishes; for instance, the spot where the fingertip presses against a bottle cap is blocked by the hand, and once a key is inserted into a lock, nothing can be seen at all.

What LingBot-Touch fills in is precisely this gap. In one sentence: vision handles the time before contact, and touch handles the time after contact.

Opening a lock with both hands is a typical example.

One hand holds the padlock while the other aligns the key with the keyhole; this step relies mainly on the eyes. But once the key is pushed in, it may tilt the lock, or get stuck halfway, requiring a slight retreat and another attempt at a different angle.

Once the key is in the keyhole, the camera is essentially of no help, and the dexterous hand can only judge by the resistance transmitted back to the fingertips through the key.

At this point, some readers may ask: why does Lingbo make touch into a separate foundation model?

The reason is that each hand has a different control method, but all of them must understand contact.

If "understanding contact" is pulled out as a separate, reusable layer, then the experience accumulated on one piece of hardware can become the starting point for the next robot.

This approach is actually in line with Lingbo's earlier work on LingBot-VLA.

VLA studies how the same brain can drive robots of different configurations—that is, "one brain, multiple machines." With Touch, the question moves one step further: can the contact experience accumulated across different hands and different sensors be learned together and then applied to new robots?

The XHand results above provide a preliminary answer.

Several design choices in LingBot-Touch were derived backward from this goal. Data is collected with gloves because gloves do not depend on the robot body, so the data scale can be expanded; the input interface is designed according to sensing principles rather than product models, in order to leave room for future new sensors.

This is also the "embodied native" approach that Lingbo has consistently emphasized: starting from how robots perceive and act in the physical world to decide how models should be designed and what data they should be trained on.

At present, LingBot-Touch has been integrated with LingBot-VLA and LingBot-Depth to improve the general-purpose brain's hand-eye coordination and ability to complete fine manipulation.

In terms of deployment, Lingbo is also conducting commercial tests in retail sorting, logistics sorting, and industrial scenarios with embodiment partners such as Leju and Tihu, as well as scenario partners such as Guoda Pharmacy and Longsheng.

Having discussed LingBot-Touch itself, let us zoom out a bit.

After GPT-6 Astra was released in September this year, the RoboDojo team had it run 42 simulated manipulation tasks under zero-shot conditions, and its overall score ranked first among 43 evaluated entries.

Frankly, this made the embodied AI community quite nervous for a while.

But in the same report, there was another set of numbers. GPT-6 Astra's success rate on the precision dimension was only 4%, while the best-performing public policy reached 20.42%; on tasks requiring precise contact, such as tube insertion and tower building, it almost completely failed, and real-robot testing was halted due to safety concerns.

The report attributed this gap to a lack of physical common sense—that is, an understanding of contact, force, and collision.

And just today, Anthropic released Claude Fable 5.2, pushing the progress bar of large models controlling robots one step further.

According to internal test results published on Anthropic's official blog, Fable 5.2 achieved an average success rate of 64.9% in the RoboDojo simulation tests, more than double that of GPT-6 Astra (29.8%) under the same test framework, and also surpassing most AI systems built specifically for controlling robots.

In the official demo, a robot controlled by Fable 5.2 can also solve a Rubik's cube, unscrew bottle caps, and draw. However, what RoboDojo tests is controlling a simulated dual-arm robot by looking at camera images.

From Fable 5.1 to Fable 5.2, over a span of more than two months, the success rate in the same test suite rose from 27.6% to 64.9%. The ability of general large models to understand tasks, reason and plan, and even directly perform operations is visibly growing stronger.

But in our view, this does not mean embodied models should step aside and only handle "acting."

There is still a gap between high scores in simulation and reliably completing work in the real world: generalization in the physical world. If the hand is changed, the material is changed, or the scenario has never been seen before, can the robot still perform well? This question cannot be avoided, and it must be tackled by a brain natively grown from the physical world.

This is also the judgment of Zhu Xing, CEO of Ant Lingbo.

He believes that the progress of digital-world large language models such as GPT-6 Astra and Claude Fable 5.2 has indeed made robots stronger in complex task reasoning and physical manipulation, and Lingbo is actively absorbing this; but to truly close the manipulation loop, "the hard nut of physical manipulation generalization still cannot be avoided," and to a large extent it still depends on breakthroughs in embodied foundation models themselves.

In his view, general large models and embodied foundation models are closer to a "1+1>2" relationship: each makes its own breakthroughs and promotes the other, jointly accelerating the maturation of the embodied intelligence industry.

LingBot-Touch is Lingbo's attempt to crack this hard nut.

Touch is a modality native to the physical world. The internet holds vast amounts of text, images, and video, yet almost no webpage has ever recorded exactly how many newtons of force a thumb applies when pinching a bottle cap. This kind of experience can only be accumulated little by little through real interaction, and it is precisely one of the foundations of generalization in the physical world.

For example, when a dexterous hand feels mahjong tiles, it relies on exactly this kind of experience.

The Two of Dots and the Three of Dots both feel like circles; to tell them apart, one must feel out how many circles there are in total and where each is arranged, and these clues are all hidden in the shallow engravings beneath the fingertip.

In the past, touch has always been scattered: each sensor has its own dataset and its own model, and switching to another hand means starting over from scratch. LingBot-Touch proves that touch can also have a unified representation.

If we push one step further, in the LingBot-Touch approach—unified coordinates, separate inputs, shared backbone, and prediction in feature space—no single link actually belongs only to touch.

This line of thinking also runs through Ant Lingbo's entire embodied-native roadmap. According to the plan, Lingbo will continue to explore the upper limits of physical manipulation generalization, build a general-purpose industry brain for robots across configurations, and promote the alignment of more physical modalities—vision, depth, touch, and even temperature—within a unified model. As for what additional sensing modalities robots may acquire in the future, no one can say for sure right now, but at least there is already a path to refer to.

This is exactly what Ant Lingbo wants to do: build a layer of intelligence between hardware and scenarios, and transform the physical experience accumulated across different bodies and different sensors into general capabilities that can be shared.

Moreover, Ant Lingbo is releasing this layer of capability in an open-source, modular way, and will continue to open-source it to the community while advancing commercialization.

LingBot-Touch can be used on its own, and it is equally effective when attached to other companies' action models such as π0.5 and Diffusion Policy; sensor manufacturers can continue along their own technical paths, and robot makers do not need to change sensors for the sake of a particular model.

Audi disclosed a figure earlier this year: across the industry, the automation rate of wiring harness production and assembly is still less than 10%. This kind of work—contacting while adjusting—is exactly where dexterous hands have long failed to take hold.

Looking at China, there are many types of dexterous hands and tactile sensors, which in the past was a headache because data could not be shared; but once experience can be shared, this diversity will instead become a data advantage.

In short, having a signal in a robot's hand does not mean it knows how to use it.

As Zhu Xing said, "the quality of modality input largely determines the quality of action output"; without good input, there is no good output.

The faster general large models run, the more embodied-native brains must not fall behind. The ability to make robots do work steadily and well in the physical world must ultimately be trained bit by bit in the real world by embodied-native brains.