Three frontline researchers discuss how GPT-6 Astra rewrites Agents, robotics, and AI pharma

Looking back at September, heavyweight news in the AI world came one after another.

On September 4, OpenAI released GPT-6 Astra. The release report did not mention a general-purpose model for robots at all, yet it suddenly topped the robot leaderboard, even rendering a batch of robot benchmarks invalid. Some joked half-seriously: the "GPT moment" the robotics field had waited years for turned out to be GPT itself.

Meta Muse gave every user a cloud computer and shot to the top of the US App Store within two weeks; competitor Instinct's valuation rose from 2.5 billion to 10 billion US dollars within a month. Personal Agents set off a new round of cash-burning customer acquisition battles, but the web pages they can open are being shut one by one.

On the AI for Science front, Claude took only 21 hours to find a new enzyme system ART whose structure resembles CRISPR. The AI world was excited, because AI has started making scientific discoveries, while the biology world's reaction was relatively muted—everyone wants to verify it in the lab first. Why this contrast?

When models evolve to a point where ordinary people can hardly feel their evolution, what do researchers feel? In this episode of Silicon Valley 101, we invite three researchers on the front lines of Agents, robotics, and AI pharma—Zhu Zheqing, founder and CEO of Pokee.ai; Xu Huazhe, founder of Pokebot and assistant professor at Tsinghua University's Institute for Interdisciplinary Information Sciences; and Wang Sheng, assistant professor at the University of Washington's Paul G. Allen School of Computer Science & Engineering—to clarify three things together: where is the commercialization of personal Agents stuck? How are robots impacted by foundation models like Astra? What is the real bottleneck of AI pharma?

Below are selected excerpts from this conversation:

Frontier scientist perspective: What has GPT-G Astra evolved?

Hongjun: After Astra's release, have you used this product? Or let's look at the online demos—what demo do you find most impressive?

Xu Huazhe: Since I work on robotics, we tested many of Astra's capabilities on robot tasks. The first thing we saw is that its adaptability is very strong. For example, give it a certain robot body and let it do the corresponding thing; it will quickly figure out that this robot moves relatively fast, or what the robot's motion space is like. For instance, if it hits a table, it will say, this table is quite tall, so should I raise my hand higher? This actually turns the reflective process capability of CoT (chain of thought) into an actual action capability, which many past robot models and large models did not have.

Another is its self-updating and self-upgrading capability. Because it has this reflection, it can do the same thing again and again, getting better each time. You've also seen this online: someone used Astra to operate a robotic arm to draw the Golden Gate Bridge. The first drawing was rather poor, but after finishing, it would think about how to draw better, and after several rounds of iteration, it could draw very realistically.

We tried many other things, such as flower arranging, a classic robot task. We added some extra difficulty: the stem of the flower was split in two, somewhat forked. At this point it would know that because it's forked, I need to pinch the fork to make it narrower, then insert it. This kind of capability is quite impressive.

Hongjun: It has actually understood some norms of operating in the physical world.

Xu Huazhe: I think essentially it still understands 3D, because this can still be understood at the level of three-dimensional dimensions, but indeed such understanding can already help with many non-dynamic physical world manipulation tasks.

Hongjun: I saw some other interesting demos online where they use Astra to generate 3D videos, very fast, and the rendering is also good.

Xu Huazhe: This is another astonishing capability of Astra in robotics: give it a few images, and it can quickly simulate and generate the images. Academia may call this Real-to-Sim. In the past, this required shooting lots of videos and photos, with a lot of manual calibration and modification. But now you give Astra such an instruction, and it quickly reproduces the entire lab or office environment for you. Then with such simulation, you can train your robot, and there will also be an improvement in simulation.

Zhu Zheqing: We are a model company. When Astra was released, it heavily emphasized its iterative capabilities in post-training, model development, etc. After trying it, it wasn't that stunning, but it did show a relatively stable improvement over previous model versions on long-horizon tasks and iteration. Compared with the original models, it has a more stable understanding of context and previously completed tasks, and doesn't hallucinate too much. So from our current experience, it's relatively good.

But there may be a few problems: the speed is relatively slower, which may also be related to the current high usage. At the same time, on longer-horizon tasks, such as letting it run for a night, hallucinations still occur. So it hasn't fundamentally solved this, but overall it has improved quite a bit.

I have friends who tested some front-end and design tasks together. In terms of front-end aesthetics and cognition, I think there is a relatively big improvement. It may be related to special training in design and aesthetics during training.

Hongjun: Bill, for this round of Astra's release, from the perspective of the model's impact on the industry, compared with the releases of opus 4.6 and 4.7 back then in terms of Coding capability evolution, is this a major node?

Zhu Zheqing: For pure coding, there may not be such a big milestone. From 4.5 to 4.6, at the time what we felt was an almost paradigm-like shift. Previously it could only be relied on for short-range coding, changes to a few files, doing some simple small projects. After the 4.6 model, basically you'd feel that tool calls almost never make mistakes anymore, and it can autonomously carry out some long-range coding tasks.

And now Astra only says it can do longer-range tasks, hallucinations can be reduced lower, and aesthetics become better. But it hasn't achieved saying that I can solve a problem that was completely unsolvable before.

Wang Sheng: I think it's a big milestone. We've now all switched from previous Claude and other series models to GPT-6 Astra to do our internal biomedical tasks. I mainly do AI for Science, and the core of that is AI for some biomedical applications.

We use Astra in two particularly useful ways: one is Computer Use, and the other is some long-range tasks.

On the Computer Use side, we use Astra to operate software written many years ago in the biomedical field. These software are used by default in the field, but they are actually all very hard to use, the interface design is not good, and there are many terms, so for people from cross-disciplinary fields, the cost of use is very high.

But Astra's Computer Use function can, directly based on some simple prompts, operate them fairly well. And I can also learn from it how to use this software. After all, in the biomedical field, many people have relatively low acceptance of AI tools. Astra may have learned in text form all the online tutorials corresponding to these software, some user logs from people who used them before, and some feedback, so it really understands how to use these software.

On the other hand are the long-range tasks. In our biomedical area, tasks are relatively complex, and these tasks sometimes last 8 to 24 hours; Astra does relatively well. Compared with some other models, it does things relatively slowly, but it reassures me quite a bit, showing that it may be more thoughtful.

We now generally switch between them: during the day we use some models with faster feedback, and when it's time to let the model run overnight, we use Astra to design a relatively long-range pipeline or prompt. This is also a point where it is relatively trustworthy, in terms of some performance in the AI for Science field.

Hongjun: I didn't expect that the Computer Use function could actually help everyone so much in the AI for Science field.

Wang Sheng: Astra did two biomedical demos at this launch. One of them is the Computer Use demo. This demo went viral in our circle; everyone discovered that you can actually directly use Astra to operate a biomedical tool. The tool it operated at the time was called MultiQC, a fairly commonly used web tool. We started using this function and found it much easier to use than imagined. Actually, I think previous models also had the Computer Use function; it's just that everyone hadn't really started thinking about how to integrate it into the pipeline of our scientific research.

Computer use revival: but commercialization is stuck at identity verification

Hongjun: Bill, could you explain in one sentence what Computer Use is?

Zhu Zheqing: Computer Use is through the model's understanding of vision, deciding which part of the screen this agent should click next, or what text to input, to complete a series of tasks. When it can chain together dozens of such actions, it can accomplish a final goal that a human wants to achieve.

Hongjun: So the difference from an API is that you don't need to provide an interface, you don't need to provide a password; it directly operates and uses the computer like a human.

Zhu Zheqing: That is, what it sees is the screen that humans see, the UI that humans see, all the visuals and interactive interfaces that humans see. It completes these tasks through the computer input and output devices that humans can use.

The Computer Use scenario shown in Astra's promotional video. Image source: OpenAI

Hongjun: I feel like Computer Use seemed to exist before too, so why did Astra's launch bring Computer Use back into popularity?

Zhu Zheqing: It feels like Computer Use is now popular again for another wave. Previously everyone thought all tools would open APIs, all tools would have agent access interfaces, and many apps and systems were doing agent native set up.

But recently, whether it's Instinct or Meta Muse, they're actually all connected to browsers behind the scenes. There's a relatively big problem, which is that many third-party apps or Amazon, for example some shopping apps, some travel apps, they are actually unwilling to open APIs.

I think this goes back to the situation with Manus back then: Manus first did Browser Use, then many others followed, and later everyone found this wasn't very efficient. Everyone went back to Coding and API calls, and now it seems some things can't be avoided, so it's back to Computer Use. So I think in the end it may still be a mixture state.

I have a relatively interesting view here, because we currently focus For relatively smaller models, for example, basically all the models we release now are around 30B, and they can run on-device and on private compute. For instance, if I have a small AI box, or an AI PC, it can run this model.

In the future, if Browser Use itself happens on your own machine rather than in the cloud, that would change the bottleneck that Browser Use itself might bring you. Because current remote Browser Use all goes through commercial machines, at most finding something like Bright Data or an IP proxy, pretending to be a consumer user, to bypass IP verification. Sites like Amazon and Expedia generally say that if it's a commercial IP, I won't let you in.

Zhu Zheqing: Luxury goods websites have been blocking them for a long time. If it uses your home network, that is, your own IP, then it won't be blocked anymore. So this could be a trend that breaks the deadlock in the future, and this could push many app vendors to change their Browser Use mechanisms, that is, this browser sandbox might happen locally rather than in the cloud.

Hongjun: This observation of yours is very interesting. A very popular consumer-facing toC application recently is Meta Muse; their positioning is to make a personal AI assistant for each person, equivalent to making a cloud-based virtual machine for everyone, and your data is stored in a cloud storage that belongs to you. According to your theory just now, it is actually still a commercial IP. If it commercializes in the future, it is very likely that merchants will block its IP.

Zhu Zheqing: In terms of the virtual machine setup behind it, Meta Muse is actually very similar to Manus at the time. It also focuses on Browser Use, because many of its use cases are related to travel and shopping. Under these use cases, you have no way to complete the task through API calls; basically in the end it all has to rely on the browser. At the time, Manus actually took exactly the same path and got completely blocked, so in the later stage, Manus, now very similar to Genspark, both focus on the white-collar work track.

If you look at Claude Code, Claude Cowork, and ChatGPT Work, in the end they all shift toward white-collar work efficiency. The core reason is that many things can be done at first, but later they are quickly all blocked by the giants.

Hongjun: For example, Amazon has now blocked Meta Muse. If more websites join in, the web pages it can access and the things it can operate will become fewer and fewer.

Zhu Zheqing: But it's impossible for them to block the IP on your computer, because then you yourself wouldn't be able to access Amazon either, so this could be an opportunity to break the deadlock. But I suspect this matter will go back and forth. On the C-end, this agent may still not be particularly ready at present. Because doing Browser Use actually places quite high demands on the model. It is hard to pull off a phenomenal scale in the short term.

Hongjun: Then do you take an optimistic view of Meta Muse's long-term development?

Zhu Zheqing: I think this wave of consumers will ultimately land in a state of combined software and hardware. I think if Meta can make wearables big, it may have a chance. The reason I think it ultimately has to land in a combination of software and hardware is that the cloud thing cannot represent the user himself, and this is also why the browser completely blocks them, because once you switch to a cloud virtual machine, it cannot verify that this person is the user himself.

I think the user's ID, or the identity of the user using the agent, will itself be key to this wave of toC. If merchants can confirm that this agent is performing this operation on behalf of this person, many merchants may be willing to do this.

But now, if a cloud virtual machine or cloud browser is used, it is very likely that there is no way to verify this matter, and they will feel that their traffic entrance has been taken away. For example, if I have a user who was originally going to visit me, and now he still visits me through an agent, then that may actually be OK for me. But if originally one user, now somehow 20 agents are created, and the spending power of these 20 agents is the same as before, and these 20 agents keep comparing prices and keep generating traffic on my side, messing up all my traffic, then for me the loss outweighs the gain.

So I think the combination of software and hardware may be an opportunity. If the hardware itself belongs to this user, it can directly confirm that this agent is this user, that it is a representative action, and then I would be willing to open this access to you.

Hongjun: Do people use xAI's Grok Bot in toB workflows?

Zhu Zheqing: The people around me who use Grok Bot are mostly trying it out, because it is relatively more proactive. But if you use Codex or Claude Code, you can actually solve these problems. Those who really integrate it into workflows are probably relatively few. Many people around me who are truly on the B-end, such as in law and healthcare, people who are truly working in those traditional enterprises, are still struggling with how to escape the quagmire of Microsoft Copilot. They can't even freely use Codex and Claude Code, let alone get to Grok at the Bot level. So I think the gap across the whole industry is now very large: those ahead can use whatever they want freely, while those behind are still struggling in the mud.

Benchmarks failing: Is the direction of model progress really right?

Hongjun: Recently I talked with researchers on relatively frontier models, and I heard a very interesting view. They believe that right now, on toC and consumer-grade agents, token usage actually doesn't cost much money. The real money-burning place now is in the core area where OpenAI, Anthropic, and DeepMind are competing — AI for Science. That is, those unsolved, undefined scientific problems; they want to solve those math problems, or need to run biological experiments like Professor Wang's, and this burns tokens extremely heavily and is internally the direction with the largest consumption.

For example, when OpenAI solved the Navier-Stokes equation problem, internally it called a stronger model, used about 10,000 sub-agents, sent about 2.7 million messages, and consumed about 130 billion output tokens. A third party calculated it at retail price, and this part of the consumption was about 22 million US dollars.

Because of the current evolution of the whole model, I think for ordinary people's perception it is no longer large. But for frontier scientists, their perception of model capability improvement is very large.

OpenAI announced it had solved the Navier-Stokes Millennium Prize problem; the solution is a vortex. Image source: OpenAI

Zhu Zheqing: At this point I'd like to ask Professor Xu and Professor Wang. I myself mainly do post training, focusing more on small models, because we want to do locally privatized models. What we can see is that models at this level, after our optimization, have an increasingly smaller gap with frontier models.

From the AI for Science perspective, or from the robotics perspective, how much improvement do you think frontier models will have in each generation of iteration?

For example, the difference from Opus 4.5 to 4.6, compared with the difference from 4.8 to Fable, or the difference from 5.6 to Astra — is this gap comparable?

Wang Sheng: My view is that in the biomedical field, its AI for Science capability has indeed improved a lot. But this improvement mainly comes from its coding ability — the general ability to code, search literature, and do long-horizon tasks — not from an improvement in its biomedical knowledge ability. Some benchmarks specifically test the Biomedical knowledge level, and Astra has not improved.

In actual use now, it is actually very hard to use Astra, or Opus, Fable, because they guard more and more strictly against biomedical tasks; they all refuse to answer, and this is a very serious problem. So internally, when we use them, we use Kimi K3 or Muse Spark 1.3. When we need to use Astra, we have to use some rather special ways to tell Astra that I am not doing a dangerous task, I am just designing a benchmark, etc. So in actual experience of using these large models, as capabilities increase, it actually gets worse and worse.

Another point is that what Bill just said makes a lot of sense, and I also agree: on various clearly defined AI for Science benchmarks, small models can indeed get closer and closer to large models. But I personally think these benchmarks themselves are not defined that well, so the direction everyone is catching up in, including the direction of large-model progress, may not be entirely correct.

When these benchmarks are defined, everyone defines one that is better for AI models to do, for example, having a clear biological dataset and predicting a clear problem, which has a very large gap from the drug discovery we actually do. Essentially these current biomedical benchmarks are just Kaggle-style data science competitions, so they degenerate into machine learning problems in MLE-bench. Then Astra's improvement in coding ability will produce great progress on the benchmark.

Xu Huazhe: I think from the robotics perspective, Astra is a leap-style improvement. In the past, whether it was Opus 4.5, 4.6, or the changes up to Fable, none brought as big a change to robotics as Astra did this time. Because all past upgrades showed almost no particularly good performance on robotics tests, and qualitatively there was no change.

But the change Astra now brings is that its whole qualitative picture has changed, and many robotics benchmarks have failed. Like RoboDojo, a relatively recognized robotics benchmark, especially for simulation, Astra has become first place. No matter which kind of data it used, the capability it brings is a leap-style upgrade.

But regarding the question of small models and large models just mentioned, I actually think that in the long term we will definitely need a small model with capabilities comparable to Astra. The reason is that robot tasks have timing requirements, need timely responses, and need highly dynamic tasks. For example, if I want to shake a towel, Astra can't do that kind of capability, because it's too slow. It knows I need to move quickly, but it can only write code to do it; if you ask it to directly output actions, there's no way to do it. In this situation, we definitely need a fast-decision model. It's just that we need to incorporate the existing Astra's CoT, or its reasoning capabilities, as well as 3D understanding, into it.

What is still missing for the GPT moment of robots?

Hongjun: So do you think the future direction of robots will become a layered architecture where large models do planning and small models are responsible for implementation and control?

Xu Huazhe: Everyone has actually always relatively agreed on the layered architecture; it's just where exactly to cut the layer? How to layer has always been debated. For example, originally everyone generally believed that VLM (vision-language model) is the task planning layer, and this task planning layer is roughly up to, I tell you I want to eat tomato scrambled eggs, then what VLM might plan is open the refrigerator door, take out what tomatoes, take out eggs, etc., and the small model below, or the embodied model, will execute how exactly to open the refrigerator door? How to crack the eggs?

But Astra's biggest change this time is that it can now do low-level motion control, equivalent to its instruction to you becoming move your hand two centimeters to the right, or put your hand on the refrigerator door handle, and if it hasn't grabbed it, it says then lean forward a bit more... So it is equivalent to pressing this layer downward.

Originally everyone's boundaries were very clear, but now it says my capabilities are very strong, and apart from being slow, I can press down a little bit. Then the thing below is more like a reflexive marionette. There will certainly be some voices saying, then can the upper-layer model in the future press down infinitely? If it presses all the way down, does it become a completely end-to-end VLM? Astra directly controls everything.

My personal view is that I think this is still not very reasonable. The first is obviously speed; the Astra control you see online all has to be watched at 16x or 24x speed. The second is that I think current Astra still lacks a lot of understanding of physics, especially the part that humans call intuition, or intuitive physics.

I may casually touch an object; if it has water it may spill, if it is glass it may break. Astra may be able to understand these things through a lot of language reasoning, but if we want it to immediately have this kind of understanding during a task, its effect may still not be good enough. This place may still need something like a world model, or a fast model, to do some intuitive reactions.

Pokebot making mapo tofu Image source: pokebot

Hongjun: You just mentioned that it is slow. Why is it slow? Is it because its model is too large?

Xu Huazhe: A large model is certainly one reason. Another is that its CoT is very long. Although it is an encrypted CoT, the CoT given out should be fake, I understand, but its internal CoT is very long, so its decision-making at each step and its thinking are very redundant, especially when facing simple tasks. We observed that when facing complex tasks and simple tasks, its thinking duration is basically similar, which also means it is very smart on complex tasks, right? But when it comes to simple tasks, it becomes an extra overhead, making it look very stupid.

Hongjun: Professor Xu, I know your company has a demo of using chopsticks to pick up small balls. What is the difference between doing it with your own model and with Astra?

Xu Huazhe: With our own model, first, it is much faster in time; second, when grasping, it continuously makes attempts, and in this process there is not much logical thinking. If I fail once at grasping, I can try from another angle.

But with Astra, after each step it takes, it has to analyze what the reason for what happened is, and based on this reason, it has to give itself instructions on what adjustment to make accordingly. This process looks more like the human thinking process, but it causes it to be very slow; second, sometimes it gets stuck in some deadlocks, sometimes it cannot understand why such a thing happened, or once its analysis is wrong, it has no way to push further.

Another point we have seen is that if we grasp an object on the tabletop, the matter of rotating it, Astra is actually relatively weaker at doing, but it can do it; it has to repeatedly try how to rotate. This kind of capability I think still has certain shortcomings at present.

Hongjun: Now robot companies are divided into many types, for example those doing brains, including how this brain is made, whether it is the Sim-to-Real path, or whether it uses data from the entire physical world; there are many schools, and also those doing bodies, and those doing data. Do you think that after Astra comes out, which type of company will be impacted more greatly?

Xu Huazhe: Let's first talk about the Sim-to-Real route. I have always believed that simulation cannot train the best robot models and policies. Many people say that Astra has revived simulation, because it does Real-to-Sim, making it very easy to transfer the real world into simulation. But it actually only makes its 3D appearance very good; however, because Astra appeared, the physics engine has not become more accurate.

Astra is definitely good news for the body and for hardware. In the two or three weeks since Astra came out, a research group at Stanford has already integrated Astra into a Unitree robot and had it do some household tasks at home. This happened very fast—it's like they wrote a paper in two weeks. You can see that once Astra has strong cross-embodiment generalization ability, if your body is of good enough quality, then perhaps connecting to Astra directly lets you do some of the work, even though it's still only demo-level.

For companies working on brains, I think there are two camps. One camp believes they will have a faster, better embodied model, and of course they may learn some things from Astra, such as the data annotation approach, the CoT approach, and so on. I think this will bring some growth to the whole industry, and I think it's a good thing. The other camp may not have strong enough embodied model capabilities, so they may just give up on this, meaning that since Astra is good, they focus on how to use it, or how to make their own things into tools for it, letting Astra call on these small things of mine to complete the task loop—that's probably more like a robot deployment company.

So I think some companies will gradually move toward being more deployment-oriented robot companies, and some companies will work on smaller capabilities like Astra, or on model training that surpasses Astra's capabilities.

Hongjun: Do you think Astra will bring some new inspiration to how research on the future of robots is done, or bring relatively big changes?

Xu Huazhe: I think it will bring very big changes. The first is the paradigm for training the entire model—I think in the future no one will ignore a paradigm like Astra's, so people training models will definitely come up with some new ideas.

The other is people who previously did traditional robot planning; they may now move toward agentic, more harness-oriented directions in robot research—how to combine various capabilities together? There is a field in robotics called TAMP (Task and Motion Planning), which puts planning and control together to do long-horizon tasks.

In the past, I think this field was relatively rigid, or rather it was a capability from the previous generation, because it often used traditional planning methods to do step-by-step planning. If my next task is to unscrew the bottle cap, then my previous step must confirm that the cap is first closed, otherwise I can't go to the next step. These things actually felt relatively outdated in the past. But when it comes to the agentic part, it seems to have regained vitality. Once you have a good model, then it comes back to planning, which may become very important. So I think it will definitely bring some changes to research as a whole.

Hongjun: You just mentioned that, for example, some particularly good robot brain companies can learn data annotation methods from Astra. What do you think is the inspiration you got from it?

Xu Huazhe: I think it's about annotating the causality and intuitive physics inside clearly: why did I succeed at this? Why did I fail? I did this thing and this result happened—the interaction process in between needs to be annotated in detail. I think this part is very important.

In the past, when many robots were annotated, they only annotated one sentence describing what you did, which is equivalent to only positive annotation, without the reasons behind it. For example, if I didn't grab it, was it because the position was off, or because the thing got wet and was too slippery, and so on. With these annotations, we can truly build a general robot brain.

Hongjun: Then does this also have an impact on the entire data annotation industry? Because I know quite a lot of people are doing robot data annotation now.

Xu Huazhe: What I've heard is that OpenAI now only buys unannotated data; it thinks annotations from outside are all wrong, and they re-annotate everything internally themselves. On the other hand, they may also be doing it for confidentiality, not wanting to reveal their annotation methods.

So for data companies there will be two kinds: one is selling raw data, raw meat, so to speak—equivalent to selling oil, a raw material, and mainly relying on volume; the other part is making cooked meat, cooked data, and then they need to understand how the most cutting-edge models are trained. If they have some knowledge in this area, they can confidently say, OK this is OpenAI's annotation method, this is Pi's annotation method, which one do you like? I'll sell it to you—the unit price can be 10 times higher, or even more.

Hongjun: Then do you think it will bring changes to the way robot research is done in the future as a whole? Because everyone used to research vertical models for robotics, and now people say the capabilities of large models are good enough and they can also drive robots—although it has various problems—so will it be that in the future robot training may still be based on a foundation model at the bottom, and it only needs to do post-training?

Xu Huazhe: If we want real deployment, a model with several T parameters is definitely very hard to deploy. If it's remote, I think it can solve some tasks, but not all tasks. If we all use foundation models like Astra, I think it's not very likely to become the model used by all robots in the future. There will definitely be a model much smaller than Astra, with similar capabilities and a similar paradigm; even if it serves in the cloud and is hosted there, I think it will still be a robot-specific model.

Hongjun: From the release of Astra to today, it's been about three or four weeks. I know that when there was the demo of a robot drawing the Golden Gate Bridge, many robot founders were very panicked, and later everyone tried it themselves. Did you have any psychological changes during the whole process?

Xu Huazhe: Yes. First, when it came out, everyone initially tried it and basically found it very impressive. The first attempt was naturally to have it grab an object, and it completed that quickly. Later, as we gradually tried more complex tasks, we felt that "the robot's GPT moment is still GPT," but looking deeper, we still saw many problems.

First, it currently can hardly do dual-arm tasks; for example, when it comes to folding clothes, Astra simply cannot fold them. So we have a feeling that on a task that is hard to describe well in language, Astra's ability is actually not very strong, so we still see that there is room in places where it needs to be sufficiently dexterous.

Second, we found that its summarization ability is not very good. For example, the first time it does something, it may take 1 hour; the second time, it may take 45 minutes; then we have it keep doing it continuously, and it is always 45 minutes, basically becoming a constant from the third time onward. It does not behave like a human or some smaller models, where after sufficient training the time gets shorter and shorter. So here we can still see that the robot's native model still has a lot of room.

And now, I think it is more about how system 1 and system 2, the fast and slow systems, should actually be combined.

Is the Astra above being called, or does it give some instructions, or what kind of instructions does it give as an interface? Because it has already appeared, you cannot avoid it. But it itself has many problems, and you cannot directly use it, so the next question is how to fully combine what robot companies are good at with models like Astra to make truly general-purpose robots. I think up to now it is still more about thinking from a technical perspective about how to do things in the future.

Hongjun: Recently there is another project that is also very popular, called Jev. It probably does a lot with a set of theories of system 1, system 2, fast thinking, and slow thinking. Do you think its impact on the robotics field is large?

Xu Huazhe: Jev positions itself as system 1, and I think it is very similar to the robot's own native model. People generally think a robot is mainly system 1, with a small portion of system 2 ability in it. So I think a robot should be a relatively fast system, but we tried Jev, and its performance on robots is very poor.

Hongjun: Why?

Xu Huazhe: For multiple reasons. Jev itself cannot take image input, so your images need to be given to it in the form of states. You must run a visual VLM or some other visual model, then give these states to it, and then let it make fast decisions. In our tests, for tasks like stacking blocks, it often cannot stack them, or after stacking they fall over. Astra can see that the first block is stacked a bit crooked, and the second block needs to be turned a bit and leaned to the other side so that it can balance. But Jev does not have such an ability.

But now I see many people saying that Jev may be usable for labeling data because it is fast. However, I just seemed to see that OpenAI is going to release a fast system, something similar to Jev, and with image multimodal input.

Hongjun: We can look forward to it. Overall, do you think Astra's release has a positive impact on the robotics industry, or an impact that shocks everyone?

Xu Huazhe: Its impact on the entire robotics field is definitely positive. For each individual, especially companies whose current model capabilities are far below Astra's, there may be some worry that I will never catch up with its capability. Overall I am still relatively optimistic. It is equivalent to a new way of thinking, and it actually illuminates many places we did not see before, so personally I think it is relatively positive.

Hongjun: Next let's talk about the AI for Science part that I am very interested in. Do you think Astra or Fable 5, among these models, is stronger in AI for Science capabilities?

Wang Sheng: I think Astra's capability is stronger, because its ability to do long-horizon tasks is a bit stronger. For our applications, it is very difficult to use existing biomedical benchmarks to measure how good a model is at biomedical tasks. If I have a benchmark that asks it to predict something, it can do very well, for example predicting the activity of a protein.

But our most important scenario now is: can an AI model produce an experimental protocol? Specifically, how should I do the experiment? At what temperature? With what molecules? At what humidity, and under what conditions should this experiment be done? This is something current models all fail to do well, but it is a very important point. I previously talked about Computer Use, that is, how AI controls computers. In our current direction, we all call it "Human Use": how AI controls some experimenters in real experiments to do experiments.

This is what we found when comparing Astra with some earlier models: Astra can still do relatively well. The corpus for these tasks is actually very small, but I think Astra's stronger capability may be able to improve this area.

Hongjun: I understand it is divided into two categories. One is AI giving a plan, predicting what structure and molecular composition your new drug might have. The second is that once the plan is made, we must verify through real experiments whether the plan is feasible. That is, now it does the first part fairly well, but in the second part, the plans it currently gives are not good enough.

Wang Sheng: I think for the first part, the thing is that we do not know whether AI is doing well or badly. There is a paradox here. If a problem is, for example, I say develop a drug to treat cancer, if I know the answer, then there is actually no need for me to research this problem.

So the questions we truly care about cannot be measured by benchmarks, and they all have no answers. So every question we care about, every question in biomedical AI for Science, ultimately needs to be validated through wet experiments.

Dry experiments are all the predictions, all the analyses simulated in a computer. Wet experiments are when we need to, in a real laboratory, in a room, through test tubes, through culture dishes, perform some operations on mouse cells and so on. So this is the difference between what can be achieved inside the computer and outside the computer.

When I give AI a problem, it is impossible for me to know the answer. GPT gives me a drug's molecular formula and says this can treat cancer. Claude gives me another molecular formula, and I have to do experiments to test it. So before doing this experiment, it is not just about giving me a molecular formula; it also needs a specific experimental plan: how can this molecular formula be synthesized? How should this small molecule be stored? On what cycle should I administer it to mice? These very detailed things are what current AI models do very poorly. It can generate a molecular formula, because there is a lot of such data in the world, but how to synthesize this molecular formula? How to give this drug to mice, and so on? They do all of these poorly. These are actually more important.

Hongjun: For the latter, looking at current large models, roughly what proportion of experiment design have they solved? How is the degree of completion?

Wang Sheng: The degree of completion is probably only 30%, and many times human experts are needed to modify it. In the field of experiment design, specialized knowledge is needed at an extremely deep level. A professor in this field may not know either; it requires technical personnel who have done many experiments specifically in this direction to be able to design an efficient, high-success-rate, and very feasible experiment.

Also, every laboratory has different equipment or devices. An AI model like Astra cannot design experiments specifically based on these devices and equipment. What I think it lacks is the ability to utilize, coordinate, and use resources in the real world. This is also what we now want to improve in the model.

Wang Sheng: I think it definitely must be equipped with a laboratory, because current general models are already very powerful, and with a laboratory it can have its own validation and accumulate its own data. There is another bottleneck in AI doing this kind of drug discovery now: its validation cycle is very long. I design a molecule, and then I use mice to do an experiment on this molecule; it inevitably takes several weeks, because mice have metabolism and will have some reactions to this drug. In these several weeks, the model may already have evolved to the next generation; AI models are evolving faster than my real experiments.

So I think this kind of frontier AI for Science research lab needs to have a wet lab, and it needs to renovate these laboratories so that these wet experiments can better cooperate with AI. For example, as soon as we start, we prepare all the cells and everything, and all the preconditions are ready; after AI predictions, the small molecules are immediately put in, so the closed loop can run faster, rather than being separate.

Anthropic's molecular biology laboratory Image source: Anthropic YouTube

Wang Sheng: Generally we first do experiments on cells and mice, which can be completed in a few weeks. After that we will do a batch of experiments, for example on primates such as monkeys; this takes at least a 4-month cycle, and sometimes we do it twice, which is 8 months. As an AI pharmaceutical company, we may not necessarily advance to human trials; we will directly license it to a large pharmaceutical company, and this licensing fee is also very considerable, generally more than 10 million US dollars, and can even be higher. A large pharmaceutical company has a patient network and can better collect patients and conduct targeted human trials. So we generally go until the monkey experiments are finished, and the cycle may be 6 to 12 months.

Hongjun: So to validate a drug, it takes much longer than for AI to design a drug. We say the whole of AI is already very smart, and it develops very fast in mathematics, because in many parts of mathematics it can be validated without experiments. Then in biology, when AI really designs a new molecule or a new drug for you, how do you determine that its design is correct?

Wang Sheng: I think this is the biggest difficulty, and this is also the fundamental reason why AI can make great achievements in mathematics and tackle the Millennium Prize Problems, but in biomedicine, up to now there is not a single drug entirely developed by AI. The feedback from the real world is too slow, and this is limited by the laws of nature; a mouse, a monkey, just needs that much time.

In the previous AlphaFold period, many AI drug discovery companies sprang up like bamboo shoots after rain. At that time it was still before large models, but most of those companies did not succeed later, because they still mainly focused on AI design and drug discovery. Even if they could shorten the cycle of AI designing a molecular formula from half a year to a few days or a few hours, they would still be limited by the later real mouse and human experiments.

So I think this is a core problem. If this problem can be solved, the explosion of AI in biomedicine and pharmaceuticals will be as huge as AI solving the Millennium mathematics problems. I think this problem cannot be fundamentally solved, but we can accelerate feedback in reality, thereby indirectly helping us solve and improve the effectiveness of AI applications in this area.

Hongjun: I heard that AlphaFold 3 open-sourced all its research, including its data. Its open source also brought a wave of biopharmaceutical companies, including companies doing this kind of vertical models and tools. Because after it was open-sourced, it is equivalent to your industry being similar to the robotics industry after Pi was open-sourced; many underlying infrastructures are already ready.

Wang Sheng: I think its open source is very important. After these tools are open-sourced, the best help to the community is that everyone pays more attention to data accumulation. Although it is open-sourced, some vertical scientific AI companies can use the data they have accumulated to maintain their moats. In fact, these data are very, very precious; one data point costs tens of thousands or hundreds of thousands of US dollars, because they are all experiments done on real mice and humans. Not only are the experiment costs high, but the experiment cycles are also long. So instead, after open-sourcing, these vertical scientific companies will have more advantages. So this is its contribution to public resources.

Hong Jun: From this perspective, will many data companies likely emerge in the biopharmaceutical industry in the future? Anyway, your model is responsible for giving ideas, and I will set up a lab specifically responsible for validation. This may also be a very good business model.

Wang Sheng: I think this is a very good business model. The current prices of AI data, for example, buying an olympiad math problem costs thousands of US dollars, and one Terminal-Bench task also costs thousands of US dollars. The newly released Terminal-Bench Science, the problems on it will also be high-value problems. I think these problems are still limited to operations within computers. In the future, if there is real real-world biological data, that value will be very high.

There is also another high-value direction, which is building an interface that can help AI conveniently conduct real biological experiments. There is now a company called Adaptyv Bio, which specifically cooperates with Anthropic. Anthropic generates an AI prediction, and in the form of MCP it immediately starts doing experiments. It has just raised 40 million US dollars.

I think there will be more and more such companies. In the future, every wet lab company will use this MCP and API form to connect directly to AI large models, thereby directly providing services to AI large models, and much of the communication cost between people will be removed. So this returns to the question I just mentioned: the most important thing is that AI itself knows what experiment to do and how to design the experiment, and wet lab companies can cooperate with it.

Hong Jun: Now, like OpenAI and also Anthropic, which of the frontier model research labs we see are equipped with their own wet labs?

Wang Sheng: Anthropic and Adaptyv Bio clearly cooperate. It recently launched a competition, inviting everyone to work on a protein design problem defined by Anthropic. It will sponsor Adaptyv Bio to do wet lab validation for everyone. So the competition recently defined by Anthropic also says that wet lab validation will be the final standard.

I have also heard a lot of news that Anthropic itself is building a wet lab team, including that some of the people it recruits also have wet lab backgrounds. So I think this is a future trend. If frontier labs really want to make high-value discoveries, it is inevitable that they need their own wet labs.

Hong Jun: Regarding the impact of models on pharmaceuticals overall, for you, what are the several major milestones? Which improvements in foundational model capabilities will make this industry suddenly have a qualitative leap or improvement?

Wang Sheng: At present, our improvement almost still depends on the improvement of Coding capabilities. Although those benchmarks for Coding tasks in Biomedical have not yet reached full marks, in my personal understanding they can almost be considered solved, because it already performs better than humans. So the improvement of Coding capabilities will solve the improvement of capabilities in dry lab work in biopharmaceuticals.

Another issue is that biomedical data is especially large and especially complex. Each person's data can reach two T. So if future models still need improvement, it is in this area: efficient processing and understanding of this two-T data. This is a very important field and also a very big opportunity.

Hong Jun: You just mentioned that you think AI has already surpassed you in some capabilities, because you are already considered a top human scientist and also do academic research. I can understand that it is no longer surpassing average intelligence, but the top-level intelligence. In this process, what problems do you think it will bring to frontier scientists like you in the future? For example, what it does may be difficult for you to verify. Will such situations exist?

Wang Sheng: Verification also depends on experimental validation. If a problem has already been well defined, with a clear test set and clear input, and it is known what problem is to be solved, then AI's ability to solve this problem will be stronger than mine. Then what human scientists can do more is define which problems AI should solve. So I think the future trend of AI in the biomedical field is no longer solving problems and designing models, but more designing benchmarks, designing evaluation methods, and defining some important problems to solve.

Also, biomedicine is related to pharmaceuticals or treating patients, directly dealing with people, so many times a scientist still needs to participate. Just as even if an airplane can fly by itself, we still want a pilot on board. AI can look by itself, treat by itself, and issue diagnoses, but I also want a doctor to really look at my report with their own eyes.

So in major decisions in scientific discovery that immediately involve a lot of money and time, a human scientist is still needed to jointly gatekeep and jointly judge the value of the problem. If it is purely solving problems, I think AI is already stronger than the strongest human scientists.

Hong Jun: You just mentioned that the definition of the entire benchmark now has problems. Then how do you think its benchmark should be defined in the future? In what direction will it evolve?

Wang Sheng: I think in the future, first of all, we need to design biomedical problems that have no upper limit, that cannot be maxed out. For example, the most advanced drugs today can kill 30% of cancer cells—can AI make better drugs that can kill 40% of cancer cells? Let AI keep designing, so there's a way with no upper limit to do this task.

What we want to design in the future is still this kind of grand, hard-to-cap task, for example, how to extend human lifespan. This kind of task with no upper limit can let AI keep solving it, and we can keep accumulating data, accumulating more and more data on a specific problem.

Hongjun: I saw Claude recently had a new result—they discovered a new enzyme system ART, which has a gene-editing tool similar to CRISPR-Cas9, but whether it has ultimately been validated is still uncertain. In the biomedical field it has already begun to have some breakthroughs. What are the signs of these breakthroughs we're seeing?

Wang Sheng: We observe that this breakthrough is almost directly proportional to the cycle of biomedical experiments—the shorter the cycle, the faster the feedback. Everyone has actually accumulated a lot of experimental data over the past few decades. In fields where experiments are done faster, models can be made better. For example, the competition Anthropic did this time was also about enzyme design—this is a field where experiments are especially fast. In the future, things like protein enzymes that can be experimented on quickly, I think will all iterate very fast. But if, for example, we want to work on aging, or study a rare disease, this kind of long-cycle problem in a specific field, then I think model progress will still be very slow.

Hongjun: What changes do enzymes and proteins bring to which fields?

Wang Sheng: In theory, proteins are omnipotent—weight loss, cancer treatment, all diseases; in theory proteins can, through some means, guide us in making drugs to treat them. Enzymes are a kind of catalyst, more about processing and accelerating various industrial chemical processes; their commercial value is very large, and they can be deployed quickly. As for proteins, if I still want to get into drugs, ultimately human trials are unavoidable, and there's still a relatively long cycle.

Do ordinary people still have value to be "distilled"?

Hongjun: I've also been talking with many model researchers. Researchers at many frontier model labs say that actually many ordinary people may now not even have the value to be distilled. That sounds very sad. Is the whole training signal now also shifting toward expert data and verifiable environments? Do you all agree with this view?

Xu Huazhe: In the robotics field, this is obviously not the case. In robotics, everyone is an expert at life—you're good at cooking, I'm good at folding clothes, he's good at walking, he's good at dancing. Of course, going further, there will also be such a stage; maybe in a year or two, you must be a dance expert, or must be a chef like Gordon Ramsay to be worth being distilled.

Wang Sheng: If knowledge is distilled, then indeed AI's knowledge is now more comprehensive, and a single scientist certainly can't compare. But some scientific discoveries and some long-term investments still need scientists or engineers there to support and endorse them, so in this respect AI cannot replace them.

At the same time, there's also a reverse distillation problem. I'm also distilling AI every day. Especially those of us doing AI for Science interdisciplinary work—there are many things I don't know, so I can quickly learn from AI. After learning, my speed of acquiring knowledge is also fast. Sometimes with some new ideas, new creativity, I can produce better scientific research results. So I think this is a process of mutual distillation.

On the other hand, from the perspective of our specific field of medicine, the role of humans is always very large. Humans not only solve problems, humans can also define problems. For example, there are some new diseases, or people want to live to 200, live to 500—these problems are endless. Then everyone has their own different physical condition, constantly providing AI with new problems, so ordinary people will absolutely not be replaced by AI.

Zhu Zheqing: From the perspective of distillation, because our current intelligence or model capability is still in very large models, which may have rote-memorized a lot and lots of knowledge, and some know-how is also forcibly added in during training. For example, when you're optimizing some benchmarks, or advancing some fixed deployment scenarios, you'll be in a state of a lot of overfitting. In this case, actually the fields we've overfit may not even be enough; there are many, many fields worth overfitting.

We all say now that models may have reached the level of general intelligence, but from the scenarios we've currently overfit, they may not be that general yet. Later, if we really want to achieve generality, we may have to jump out of distilling each scenario and enter a state of simulating a person learning from scratch. From this angle, the ability we actually need to distill is not whether I can do this thing well, but distilling how humans learn, and how to learn from scratch even faster than humans.

Hongjun: Today is rare—we've invited three entrepreneurs and scholars from three different fields, whom I think are doing well. Would you all like to say, in your field now, what is the one issue you care about and pay most attention to?

Zhu Zheqing: The issue we are most focused on right now is actually that, on current hardware, storage worldwide is very limited, memory is very expensive, and VRAM is very expensive. Many people also worry: is all data being moved to the cloud? If in the future we believe all computing and compute power will be spread out, everyone will have their own compute power, and every company will have its own compute power. Compute power will no longer be concentrated only in data centers. Can we imagine that everyone can have their own compute power and run their own models? These models do not necessarily have to be the largest. Can we use these models to solve 80 to 90 percent of the problems in life? To solve the problem that AI currently cannot be popularized completely and without limit?

I actually still agree that future models need to develop toward being larger and stronger. For example, in AI for Science, I may need a model with a very high degree of depth. In normal settings, such as enterprise work and personal life, you may need a model with a high degree of breadth, and this model does not need to be very large. So this is a question of combination. If the right combination is done, it can actually solve the current bottlenecks in hardware demand and electricity demand.

Xu Huazhe: From the robotics perspective, what we care about most is certainly zero-shot generalization ability. Previous robots actually had two paths: one is deployment, and the other is generalization. These two paths are actually somewhat orthogonal. For deployment, it is a bit like AI empowering the previous generation of robots. A robotic arm in my factory may originally have tightened screws in an open loop; open loop means I do not take feedback. Now that I have AI, I can tighten screws in a closed loop, or load goods in a closed loop. But from my perspective, this matter is still somewhat against the times. You still need five PhD-level people to sleep in the factory for three months to solve all these issues, such as the robot's stability, speed, and whether to customize the end effector gripper.

But today when we talk about Astra, I think ultimately we still hope for a general-purpose robot. It can achieve rapid deployment zero-shot. This deployment should be such that five arbitrary people who are not very strong are placed there. Such zero-shot generalization will certainly have redundancy. Is such redundancy acceptable to us? I think it is acceptable. In this case, whether there can be a sufficiently good and fast zero-shot generalization model is still the main pursuit of robotics.

Hongjun: How far do you think robots are from that step?

Xu Huazhe: I think early 2028 will be a time point when general-purpose robots begin to become truly practical. Starting now, perhaps 18 months later, everyone can, at home and in the form of language, have a robot do 90 percent of the things at home.

Hongjun: This prediction is quite aggressive.

Xu Huazhe: If you have played with Astra, and if next month perhaps you can play with our model, you should feel that this prediction is still relatively reasonable.

Wang Sheng: For AI for Science, if there is really to be a breakthrough, the most important thing is AI's understanding of the real world, and the use and allocation of real-world resources, and so on. Just as when we study science, the essence is to study the real world: physics studies the universe, biology studies life, and chemistry studies the microscopic world.

But this does not mean I design a robot to replace humans and automatically conduct experiments. Rather, it means AI knows: according to what resources are available in my laboratory now, what should I have this robot, or this person, do? For example, in the future when I prompt AI, I will not say to it, "Design me a drug that can treat cancer," but rather, "I give you 100,000 US dollars; design me a drug, and I have these experiments that can be done," for example, a mouse experiment costs 2,000 US dollars, and a human trial may require some kind of cycle.

AI can truly plan such super-complex tasks in the real world. These are rarely seen in its corpus. This may require a new paradigm. Just as now we all use AI as humans, and humans study how to write good prompts, how to direct AI well, how to write good prompts, and how to build good agents; in the future, perhaps AI will need to write clear, executable instructions, and AI will prompt the real world, very efficiently, very clearly, and without ambiguity. Humans or robots can, according to these instructions, use resources in the real world to complete these things efficiently.

Just as before we all said that judging whether AI can follow human instructions matters, in the future we may need instruction generation: whether it can generate better instructions to help us mobilize real-world resources. We may even make some transformations to the real world: some future experimental equipment and instruments will not be for humans to look at or for humans to use, but for AI to use. Now, for example, when taking X-rays or CT scans, we still need to synthesize images so that human doctors can see them; in the future, perhaps the underlying waveforms or signals can be taken directly and shown directly to AI, skipping the step of generating images.

So this is what I think is most important: AI can better enter the real world, whether through robots or through a real human, but it must know how to use real-world resources to solve such extremely difficult problems.

This article comes from the WeChat public account "Silicon Valley 101", author: Hongjun, published by 36Kr with authorization.