Agent, robots, and pharma: who will be rewritten first by GPT-6 Astra?

By Silicon Valley 101

Looking back at September, the AI world saw one major news story after another.

On September 4, OpenAI released GPT-6 Astra. The release report did not mention a general model for robots at all, yet it immediately shot to the top of robot benchmarks, even rendering a batch of robot benchmarks directly invalid. Some joked half-seriously: the "GPT moment" the robotics field had waited years for turned out to be GPT itself.

Meta Muse gave every user a cloud computer and shot to the top of the US App Store within two weeks; competitor Instinct's valuation rose from 2.5 billion to 10 billion US dollars within a month. Personal Agents set off a new round of cash-burning customer acquisition battles, but the web pages they can open are being shut one by one.

On the AI for Science front, Claude took only 21 hours to find a new enzyme system called ART whose structure resembles CRISPR. The AI world was excited, because AI has started making scientific discoveries, while the biology world's reaction was relatively muted—everyone wants to verify it in the lab first. Why this contrast?

When models evolve to a point where ordinary people can hardly feel their evolution, what does it feel like to researchers? In this episode of Silicon Valley 101, we invited three researchers on the front lines of Agents, robotics, and AI drug discovery—Zhu Zheqing, founder and CEO of Pokee.ai; Xu Huazhe, founder of Pokebot and assistant professor at Tsinghua University's Institute for Interdisciplinary Information Sciences; and Wang Sheng, assistant professor at the University of Washington's Paul G. Allen School of Computer Science & Engineering—to clarify three things together: where is the commercialization of personal Agents stuck? How are robots being impacted by foundation models like Astra? What is the real bottleneck in AI drug discovery?

The following are highlights from this conversation:

Hongjun: After Astra was released, have you used this product? Or let's look at the online demos—what demo did you find most impressive?

Xu Huazhe: Since I work on robotics, we tested many of Astra's capabilities on robotic tasks. The first thing we saw is that its adaptability is very strong. For example, give it a certain robot body and let it do the corresponding task, and it will quickly figure out that this robot moves relatively fast, or what the robot's motion space is like. For instance, if it hits a table, it will say, this table is quite high, so should I raise my hand a bit higher? This actually turns the reflective process of CoT (chain of thought) into a capability for actual actions, which many past robot models and large models did not have.

Another is its ability to self-update and self-upgrade. Because it has this kind of reflection, it can do the same thing again and again, getting better each time. You've also seen this online: someone used Astra to operate a robotic arm to draw the Golden Gate Bridge. The first drawing was rather poor, but after finishing, it would think about how to draw better, and after several rounds of iteration, it could draw it very realistically.

We tried many other things, such as flower arranging, a classic robotic task. We added some extra difficulty: the stem of the flower was split in two, somewhat forked. At this point it would know that because it is forked, I need to pinch the fork to make it narrower, then insert it. This kind of capability is quite impressive.

Hongjun: It has actually understood some norms of operating in the physical world.

Xu Huazhe: I think fundamentally it still understands 3D, because this can still be understood at the level of three-dimensional dimensions, but indeed such understanding can already help with many non-dynamic physical world manipulation tasks.

Hongjun: I saw some other interesting demos online where they use Astra to generate 3D videos, very fast, and the rendering is also very good.

Xu Huazhe: This is another astonishing capability of Astra in robotics: give it a few images, and it can quickly simulate and generate the images. Academia might call this Real-to-Sim. In the past, this required shooting a lot of videos and photos, and a lot of manual calibration and modification. But now you give Astra such an instruction, and it quickly reproduces the entire laboratory or office environment for you. Then with such simulation, you can train your robot, and there will also be an improvement in simulation.

Zhu Zheqing: We are a model company. When Astra was released, it heavily emphasized its iterative capabilities in post-training, model development, and so on. After trying it, it wasn't that stunning, but it did show a relatively stable improvement over previous model versions on long-horizon tasks and iteration. Compared with the original models, it has a more stable understanding of context and previously completed tasks, and does not hallucinate too much. So from our current experience, it is relatively good.

But there may be a few problems: the speed is relatively slower, which may also be related to the current high usage. At the same time, on longer-horizon tasks, such as letting it run overnight, hallucinations still occur. So it has not fundamentally solved this, but overall it has improved quite a bit.

I have friends who tested some front-end and design tasks together. In terms of front-end aesthetics and cognition, I think there is a relatively big improvement. It may be related to special training in design and aesthetics during training.

Hongjun: Bill, for this round of Astra's release, from the perspective of the model's impact on the industry, compared with the releases of opus 4.6 and 4.7 back then in terms of the evolution of Coding capabilities, is this a major milestone?

Zhu Zheqing: For pure coding, it may not be such a big milestone. From 4.5 to 4.6, at the time what we felt was an almost paradigm-like shift. Previously we could only rely on it for short-range coding, changes to a few files, and some simple small projects. After the 4.6 model, you basically felt that tool calls almost never went wrong, and it could autonomously carry out longer-range coding tasks.

Now Astra only says it can handle longer-range tasks, hallucinations can be reduced further, and aesthetics have become better. But it has not achieved the point where I can solve a problem that was completely unsolvable before.

Wang Sheng: I think it is a big milestone. We have now all switched from previous models such as the Claude series to GPT-6 Astra to do our internal biomedical tasks. I mainly work on AI for Science, and the core of that is AI for some biomedical applications.

There are two particularly useful things about using Astra: one is Computer Use, and the other is some long-range tasks.

In terms of Computer Use, we use Astra to operate software in the biomedical field that was written many years ago. These software tools are used by default in the field, but they are actually very hard to use, their interfaces are poorly designed, and there are many terms, so for people from other fields the cost of use is very high.

But Astra's Computer Use function can, directly based on some simple prompts, operate them fairly well. And I can also learn from it how to use this software. After all, in the biomedical field, many people have relatively low acceptance of AI tools. Astra may have learned, in text form, all the online tutorials corresponding to these software tools, some user logs from people who used them before, and some feedback, so it really understands how to use these software tools.

The other aspect is long-range tasks. In our biomedical area, tasks are relatively complex, and these tasks sometimes last 8 to 24 hours. Astra does relatively well. Compared with some other models, it is slower, but it reassures me quite a bit, which shows it may be more thoughtful.

We now generally switch between them: during the day we use some models with faster feedback, and when we are ready to let the model run overnight, we use Astra to design a relatively long-range pipeline or prompt. This is also a relatively trustworthy aspect of its performance in the field of AI for Science.

Hongjun: I did not expect that the Computer Use function could actually help everyone so much in the AI for Science field.

Wang Sheng: At this release, Astra made two biomedical demos. One of them is the Computer Use demo. This demo went viral in our circle. Everyone discovered that Astra could actually be used directly to operate a biomedical tool. At the time, one tool it operated was called MultiQC, a fairly commonly used web tool. We started using this function and found it much easier to use than we had imagined. Actually, I think previous models also had the Computer Use function; it is just that everyone had not really started thinking about how to integrate it into our scientific research pipeline.

Hongjun: Bill, could you explain in one sentence what Computer Use is?

Zhu Zheqing: Computer Use uses the model's understanding of vision to decide which part of the screen this agent should click next, or what text to enter, in order to complete a series of tasks. When it can chain together dozens of actions, it can accomplish a final goal that humans want to achieve.

Hongjun: So the difference between it and an API is that you do not need to provide an interface or a password; it directly operates and uses the computer like a human.

Zhu Zheqing: What it sees is the screen that humans see, the UI that humans see, and all the visual and interactive interfaces that humans see. It completes these tasks through the computer input and output devices that humans can use.

The Computer Use scenario shown in Astra's promotional video. Image source: OpenAI

Hongjun: I think Computer Use seemed to exist before as well, so why did Astra's release bring Computer Use back into the spotlight?

Zhu Zheqing: It feels like Computer Use has become popular again now. Previously everyone thought all tools would open APIs, all tools would have agent access interfaces, and many apps and systems were doing agent native setups.

But recently, whether it is Instinct or Meta Muse, they actually all connect to a browser behind the scenes. A relatively big problem is that many third-party apps, or Amazon, or for example some shopping apps and some travel apps, are actually unwilling to open their APIs.

I think this goes back to the situation with Manus back then. Manus first did Browser Use, then many others followed, and later everyone found that this was not very efficient. Everyone went back to coding and API calls. Now it seems some things cannot be avoided, and we are back to Computer Use, so I think in the end it may still be a mixture.

I have a relatively interesting view here, because we currently focus on relatively smaller models—for example, basically all the models we release now are around 30B—they can run on-device and on private compute. For instance, if I have a small AI box, or I have an AI PC, it can run this model.

In the future, if Browser Use itself happens on your machine rather than in the cloud, that will change the bottleneck that Browser Use itself may bring you. Because current remote Browser Use all goes through commercial machines, at most finding something like Bright Data or an IP proxy to pretend to be a consumer user and bypass IP verification. Sites like Amazon and Expedia generally say that if it is a commercial IP, I won't let you in.

Zhu Zheqing: Luxury goods websites have been blocking this for a long time. If it uses your home network, that is, your own IP, then it won't be blocked anymore. So this may be a trend that breaks the deadlock in the future, and this matter may push many app vendors to change their Browser Use mechanisms—that is, the browser sandbox may happen locally rather than in the cloud.

Hongjun: This observation of yours is very interesting. A very popular toC consumer application recently is Meta Muse. Their positioning is to build a personal AI assistant for everyone; it is equivalent to giving each person a cloud-based virtual machine, and your data is stored in storage that belongs to you in the cloud. According to your theory just now, it is actually still a commercial IP. If it becomes commercialized in the future, it is very likely that merchants will block its IP.

Zhu Zheqing: In terms of the virtual machine setup behind it, Meta Muse is actually very similar to Manus at the time. It also focuses on Browser Use, because many of its use cases are related to travel and shopping. Under these use cases, you have no way to complete the task through API calls; basically it all has to rely on the browser in the end. At the time, Manus actually took exactly the same path and was completely blocked, so later on Manus, now very similar to Genspark, both focus on the white-collar work suite.

If you look at Claude Code, Claude Cowork, and ChatGPT Work, in the end they all shift toward white-collar work efficiency. The core reason is that many things could be done at first, but later they were quickly all blocked by the giants.

Hongjun: For example, Amazon has now blocked Meta Muse. If more websites join in, the fewer web pages it may be able to access and the fewer things it may be able to operate.

Zhu Zheqing: But it is impossible for them to block the IP on your computer, because then you yourself wouldn't be able to get onto Amazon either, so this may be an opportunity to break the deadlock. But I suspect this matter will go back and forth. On the C-end, this agent may still not be particularly ready. Because doing Browser Use actually places quite high demands on the model. It is hard to pull off a phenomenal scale in the short term.

Hongjun: Then would you look at the long-term development of something like Meta Muse?

Zhu Zheqing: I think this wave of consumers will ultimately land in a state of combined software and hardware. I think if Meta can make wearables big, it may have a chance. The reason I think it ultimately has to land in combined software and hardware is that the cloud cannot represent the user himself, and this is also why browsers completely block them, because once you switch to a cloud-based virtual machine, it cannot verify that this person is the user himself.

I think the user's ID, or the identity of the user using the agent, will itself be key to this wave of toC. If merchants can confirm that this agent is performing this operation on behalf of this person, many merchants may be willing to do this.

But now, if you use a cloud-based virtual machine and a cloud-based browser, it is very likely that they have no way to verify this matter, and they will feel that their traffic gateway is being taken away from them. For example, if I have a user who was originally going to visit me and now still visits me through an agent, then that may actually be OK for me. But if originally there was one user, and now there are 20 agents, and the spending power of these 20 agents is the same as before, and these 20 agents keep comparing prices and keep generating traffic on my side, messing up all my traffic, then for me the loss outweighs the gain.

So I think combined software and hardware may be an opportunity. If the hardware itself belongs to this user, it can directly confirm that this agent is this user and that this is a representative action, and then I would be willing to open access to you.

Hongjun: Does everyone use xAI's Grok Bot in toB workflows?

Zhu Zheqing: The people around me who use Grok Bot are mostly trying it out, because it is relatively more proactive. But if you use Codex or Claude Code, you can actually solve these problems. Those who really integrate it into their workflows may be relatively few. Many people around me who are truly on the B-side, such as in law and healthcare, people who are truly working in those traditional enterprises, are still struggling with how to escape the quagmire of Microsoft Copilot. They can't even freely use Codex and Claude Code, let alone get to the level of Grok Bot. Bot level. So I think the gap across the industry is now very large: those at the front can use whatever they want, while those behind are still struggling in the mud.

Hongjun: Recently I talked with researchers working on more frontier models, and I heard a very interesting view. They believe that right now, for toC and consumer-grade agents, token usage actually does not cost much money. The real money-burning area is the core field where OpenAI, Anthropic, and DeepMind are competing — AI for Science. That is, those unsolved and undefined scientific problems; they want to solve those math problems, or they need to run biological experiments like Professor Wang's. This burns a huge, huge amount of tokens, and it is the direction with the largest internal consumption.

For example, when OpenAI solved the Navier-Stokes equations problem, internally it called a stronger model, used about 10,000 sub-agents, sent about 2.7 million messages, and consumed about 130 billion output tokens. A third party calculated it at retail price, and this part of the consumption was about 22 million US dollars.

Because with the current evolution of the whole model, I think for ordinary people's perception this is no longer a big deal. But for frontier scientists, their perception of the improvement in model capabilities is very large.

OpenAI announced that it had solved the Navier-Stokes Millennium Prize Problem; the solution is a vortex. Image source: OpenAI

Zhu Zheqing: Here I would like to ask Professor Xu and Professor Wang a question. I myself mainly do post training, focusing more on small models, because we want to build locally privatized models. What we can see is that models at this level, after our optimization, have an increasingly smaller gap with frontier models.

From the perspective of AI for Science, or from the perspective of robotics, how much improvement do you think frontier models will have in each generation of iteration?

For example, the difference from Opus 4.5 to 4.6, compared with the difference from 4.8 to Fable, or the difference from 5.6 to Astra — is this gap comparable?

Wang Sheng: My view is that in the biomedical field, its AI for Science capability has indeed improved a lot. But this improvement mainly comes from its coding ability — the general ability to code, search the literature, and do long-horizon tasks — not from an improvement in its biomedical knowledge capability. Some benchmarks specifically test the Biomedical knowledge level, and Astra has not improved there.

In actual use now, it is actually very difficult for us to use Astra, or Opus or Fable, because they guard more and more strictly against biomedical tasks; they will refuse to answer, and this is a very serious problem. So internally, when we use them, we use Kimi K3 or Muse Spark 1.3. When we do need to use Astra, we have to use some rather special ways to tell Astra that I am not doing a dangerous task, I am just designing a benchmark, and so on. So in practice, the user experience of these large models, as their capabilities increase, is actually getting worse and worse.

Another point is that what Bill just said makes a lot of sense, and I also agree: on various clearly defined AI for Science benchmarks, small models can indeed get closer and closer to large models. But I personally think these benchmarks themselves are not defined well enough, so the direction everyone is chasing, including the direction in which large models are improving, may not be entirely correct.

When these benchmarks are defined, everyone defines them in a way that makes it easier for AI models to do them — for example, there is a clear biological dataset and a clear problem to predict. This has a very large gap from the drug discovery we actually do. Essentially, today's biomedical benchmarks are just Kaggle-style data science competitions, so they degenerate into machine learning problems in MLE-bench. Then an improvement in coding ability like Astra's will produce great progress on the benchmark.

Xu Huazhe: I think from the robotics perspective, Astra is a leapfrog improvement. In the past, whether it was Opus 4.5, 4.6, or the change to Fable, none brought as big a change to robotics as Astra did this time. Because all past upgrades showed almost no particularly good performance on robotics tests, and there was no qualitative change.

But the change Astra brings now is that the whole thing has changed qualitatively, and many robotics benchmarks have become invalid. For example, RoboDojo, a relatively recognized robotics benchmark, especially for simulation, Astra has become number one. No matter what kind of data it used, the capability it brings is a leapfrog upgrade.

But regarding the question of small models and large models that we just mentioned, I actually think that in the long term we will definitely need a small model with capabilities comparable to Astra. The reason is that robot tasks have timing requirements, need timely responses, and need highly dynamic tasks. For example, if I want to shake a towel, Astra can't do that kind of capability, because it's too slow. It knows I need to move quickly, but it can only write code to do it; if you ask it to directly output actions, there's no way to do that. In this situation, we definitely need a fast decision-making model. It's just that we need to incorporate the existing Astra's CoT, or its reasoning ability, as well as 3D understanding, into it.

Hongjun: So do you think the future direction of robots will become a layered architecture where large models do planning and small models handle implementation and control?

Xu Huazhe: Everyone has actually relatively agreed on the layered architecture; it's just where exactly to draw the line of the layering, and what kind of layering approach to use, that people have been debating. For example, originally everyone generally believed that the VLM (vision-language model) was the task planning layer, and this task planning layer roughly went as far as: I tell you I want to eat tomato and egg stir-fry, and what the VLM might plan is to open the fridge door, take out tomatoes, take out eggs, and so on, while the small model underneath, or the embodied model, would execute how exactly to open the fridge door, how to crack the eggs, and so on.

But the biggest change in Astra this time is that it can now do low-level motion control, equivalent to its instructions to you becoming: extend your hand two centimeters to the right, or put your hand on the fridge door handle; if it hasn't grabbed it, it says then lean forward a bit more... So it's equivalent to pushing this layer downward.

Originally everyone's boundaries were very clear, but now it says its capability is very strong, and apart from being slow, it can push down a little bit. Then the thing underneath is more like a reflexive marionette. There will certainly be some voices saying, then can the upper-layer model in the future push down infinitely? If it pushes all the way down, does it become a fully end-to-end VLM? Astra directly controls everything.

My personal view is that I think this is still not very reasonable. The first is obviously speed—the Astra control you see online has to be played at 16x or 24x speed to watch. The second is that I think the current Astra still lacks a lot of understanding of physics, especially the part that humans call intuition, or intuitive physics.

I might casually touch an object; if it has water it might spill, if it's glass it might break. Astra may be able to understand these things through a lot of language reasoning, but if we want it to immediately have this kind of understanding during a task, its effect may still not be good enough. This place may still need something like a world model, or a fast model, to do some intuitive reactions.

Pokebot making mapo tofu. Image source: pokebot

Hongjun: You just mentioned that it's slow. Why is it slow? Is it because its model is too large?

Xu Huazhe: The model being large is certainly one reason. Another is that its CoT is very long. Although it's an encrypted CoT, the CoT it gives out should be fake, I understand, but its internal CoT is very long, so every step of decision-making it does involves very redundant thinking. Especially when facing simple tasks—we've observed that when facing complex tasks and simple tasks, its thinking duration is basically similar, which also means it's very smart on complex tasks, right? But when it comes to simple tasks, it becomes an extra overhead, making it look very dumb.

Hongjun: Professor Xu, I know your company has a demo of using chopsticks to pick up small balls. What's the difference between doing it with your own model and doing it with Astra?

Xu Huazhe: With our own model, first, it's much faster in time; second, when grabbing, it continuously makes attempts, and in this process there isn't much logical thinking—if I grab once and fail, I can try from a different angle.

But if it's Astra, after every step it takes, it has to analyze what the reason for what happened is, and based on this reason, it has to give itself instructions on what corresponding adjustment to make. This process looks more like the human thinking process, but it causes it to be very slow; second, it sometimes gets stuck in some deadlocks—sometimes it can't understand why such a thing happened, or once its analysis goes wrong, it has no way to push further.

There's another point we've seen: if we grab an object on a tabletop, the matter of rotating it—Astra is actually relatively weaker at this, but it can do it; it has to repeatedly try how to rotate it. This kind of capability, I think, still has certain shortcomings at present.

Hongjun: Now robot companies are divided into many types, for example those doing brains, including how to do this brain—whether it's the Sim-to-Real path, or whether it uses data from the entire physical world; there are many schools, and also those doing bodies, and those doing data. Do you think that after Astra comes out, which type of company will be impacted the most?

Xu Huazhe: Let's first talk about the Sim-to-Real route. I have always believed that simulation cannot train the best robot models and policies. Many people say that Astra has revived simulation, because it does Real-to-Sim, making it very easy to transfer the real world into simulation. But it actually only makes its 3D appearance very good; it's not that because Astra appeared, the physics engine became more accurate.

Astra is definitely good for the body and good for hardware. In the two or three weeks since Astra came out, a research group at Stanford has already integrated Astra into a Unitree robot and had it do some household tasks at home. This happened very fast—it's like they wrote a paper and got it done in two weeks. You can see that once Astra has strong cross-embodiment generalization ability, if your body is of sufficiently good quality, then connecting to Astra may directly let you do part of the work, even though it can still only be demo-level.

For companies working on brains, I think there are two camps. One camp believes it will have a faster, better embodied model, and of course it may learn some things from Astra, such as data annotation methods, CoT methods, and so on. I think this will bring growth to the entire industry, and I think it's a good thing. The other camp may not have strong enough embodied model capabilities, so it may simply give up on this, meaning: since Astra is good, I'll focus on how to use it, or on how to make my own thing a tool for it, letting Astra call my little components to complete the task loop—that would be more like a robot deployment company.

So I think some companies will gradually move toward being more deployment-oriented robot companies, and some companies will work on smaller capabilities like Astra, or on model training that surpasses Astra's capabilities.

Hongjun: Do you think Astra will bring some new inspiration to how research on the future of robots as a whole is done, or bring relatively big changes?

Xu Huazhe: I think it will bring very big changes. The first is the paradigm for training the entire model—I think everyone will not ignore a paradigm like Astra in the future, so people who train models will definitely generate some new ideas.

The other is people who previously did traditional robot planning; they may now move toward agentic, more harness-oriented directions in robot research—how to combine all kinds of capabilities together? There is a field in robotics called TAMP (Task and Motion Planning), which puts planning and control together to do long-horizon tasks.

In the past, I think this field was rather rigid, or rather it was a capability of the previous generation, because it often used traditional planning methods to plan step by step. If my next task is to unscrew the bottle cap, then my previous step has to confirm that the cap is first closed; otherwise I can't move to the next step. These things actually felt relatively outdated in the past. But when it comes to the agentic part, it seems to have regained vitality. Once there is a good model, then coming back to planning may become very important. So I think it will definitely bring some changes to research as a whole.

Hongjun: You just mentioned that, for example, some particularly good robot brain companies can learn data annotation methods from Astra. What do you think you have gained inspiration from in this?

Xu Huazhe: I think it is about annotating the causality and intuitive physics inside clearly: why did I succeed at this? Why did I fail? I did this thing, and this result happened—the interaction process in between needs to be annotated in detail. I think this part is very important.

In the past, when many robots were annotated, they only annotated one sentence describing what you did, which is equivalent to only doing positive annotation, without the reasons behind it. For example, I didn't grasp it—was it because the position was off, or because the thing was wet and too slippery, and so on. With these annotations, we can truly build a general-purpose robot brain.

Hongjun: Then does this also have an impact on data annotation companies as a whole? Because I know quite a lot of people are doing robot data annotation now.

Xu Huazhe: The rumor I heard is that OpenAI now only buys unannotated data; it thinks all outside annotation is wrong, and they re-annotate everything internally themselves. On the other hand, they may also be doing it for confidentiality, not wanting to reveal their annotation methods.

So for data companies there will be two types. One is selling raw data, like raw meat—it's equivalent to selling oil, a raw material, and mainly relies on volume. The other part is making cooked meat, cooked data—then they need to understand how the most cutting-edge models are trained. If they have some knowledge in this area, they can confidently say, OK, this is OpenAI's annotation method, this is Pi's annotation method, which one do you like? I'll sell it to you. Their unit price can be 10 times higher, or even more.

Hongjun: Then do you think it will bring changes to the way robot research as a whole is done in the future? Because everyone used to be researching vertical models for robotics, and now it's said that large models' capabilities are good enough and they can also drive robots—although they have all kinds of problems—so will robot training in the future perhaps still be based on a foundation model at the bottom, and it only needs to do post-training?

Xu Huazhe: If we want true deployment, a multi-trillion-parameter model is definitely very hard to deploy. If it's remote, I think it can solve part of the tasks, but not all tasks. If we all use foundation models like Astra, I think it's not very likely to become the model used by all robots in the future. There will definitely be a model much smaller than Astra, with similar capabilities and the same paradigm; even if it serves in the cloud and is hosted there, I think it will still be a robot-specific model.

Hongjun: Do you think that from the release of Astra to today, it's already been about three or four weeks. I know that when there was that demo of a robot drawing the Golden Gate Bridge, many robot founders were very panicked, and later everyone tried it themselves. Over the whole process, have you had some psychological changes?

Xu Huazhe: Yes. First, when it first came out, everyone was just starting to try it, and basically everyone found it very impressive. At first, of course, we tried having it grab an object, and it completed that quickly. Later, as we gradually tried more complex tasks, we felt that "the robot's GPT moment is still GPT," but looking deeper, we still saw many problems.

First, its dual-arm tasks are almost impossible to do right now. For example, when it comes to folding clothes, Astra simply cannot fold them at all. So we have the feeling that on tasks that are hard to describe well in language, Astra's ability is actually not very strong, so we still see that there is room for it to improve in places that require enough dexterity.

Second, we found that its summarization ability is not very good. For example, the first time it does something, it might take 1 hour; the second time, it might take 45 minutes; then we have it keep doing it continuously, and it is always 45 minutes—basically from the third time onward it becomes a constant. It won't be like a human or some smaller models, where after sufficient training the time gets shorter and shorter. So here we can still see that the robot's own native model still has a lot of room for improvement.

And now, I think it's more about how system 1 and system 2, the fast and slow systems, should actually be combined.

Should Astra above be called, or should it give some instructions, or what kind of instructions should it give as an interface? Because it has already appeared, you can't avoid it. But it itself has many problems, and you can't directly use it, so the next question is how to fully combine what robot companies are good at with models like Astra to make truly general-purpose robots. I think that up to now it's still more about thinking from a technical perspective about how to do things in the future.

Hongjun: There's another project that has been very popular recently, called Jev. It probably mostly uses a set of theories of system 1, system 2, fast thinking, and slow thinking. Do you think its impact on the robotics field is big?

Xu Huazhe: Jev positions itself as system 1, and I think that is very similar to the robot's own native model. People generally think of a robot as mainly system 1, with a small portion of system 2 capability inside. So I think a robot should be a relatively fast system, but we tried Jev, and its performance on robots is very poor.

Hongjun: Why?

Xu Huazhe: For various reasons. Jev itself cannot take image input, so your images need to be given to it in the form of states. You have to run a visual VLM or some other visual model, then give these states to it, and then let it make rapid decisions. In our tests, for tasks like stacking blocks, it often just can't stack them, or after stacking they fall over. Astra can see that the first block is stacked a bit crooked, and the second block needs to be turned a bit and leaned toward the other side, so that it can balance. But Jev does not have this kind of ability.

But now I see many people saying that Jev might be usable for labeling data, because it is fast. However, I just seemed to see that OpenAI is going to release a fast system, something similar to Jev, and with image multimodal input.

Hongjun: We can look forward to it. So overall, do you think Astra's release has a positive impact on the robotics industry, or an impact that shocks everyone?

Xu Huazhe: Its impact on the entire robotics field is definitely positive. For each individual, especially companies whose current model capabilities are far below Astra's, there may be some worry that I will never catch up to its capability. Overall I am still relatively optimistic. It is equivalent to a new way of thinking, and it has actually illuminated many places we had not seen before, so personally I think it is on the positive side.

Hongjun: Next, let's talk about the AI for Science part that I am very interested in. Do you think Astra or Fable 5, among these models, is stronger in AI for Science capabilities?

Wang Sheng: I think Astra's ability is stronger, because its ability to do long-horizon tasks is a bit stronger. For our applications, it is very hard to use existing biomedical benchmarks to measure how good a model is at biomedical tasks. If I have a benchmark that asks it to predict something, it can do very well, for example predicting the activity of a protein.

But our most important scenario right now is: can an AI model produce an experimental plan? Specifically, how should I do the experiment? At what temperature? With what molecules? At what humidity, and under what conditions should this experiment be done? This is something that current models all do poorly, but it is very important. I previously talked about Computer Use, that is, how AI controls computers. In our current direction, we all call it "Human Use": how AI controls some experimenters in real experiments to do experiments.

This is what we found when comparing Astra with some earlier models: Astra can still do relatively well. The corpus for these tasks is actually very small, but I think Astra's stronger capabilities may be able to bring some improvement in this area.

Hongjun: As I understand it, it is divided into two categories. One is AI giving a plan, predicting what structure and molecular composition your new drug might have. The second is that once the plan is made, we must verify through real experiments whether the plan is feasible. Right now it does the first part fairly well, but in the second part, the plans it gives are still not good enough.

Wang Sheng: I think for the first part, actually we don't know whether the AI is doing well or badly. There is a paradox here. If a problem is, for example, developing a drug to treat cancer, if I already know the answer, then there is actually no need for me to research this problem.

So the questions we truly care about cannot be measured by benchmarks, and they all have no answers. So every question we care about, every biomedical AI for Science question, ultimately needs to be verified through wet lab experiments.

Dry lab means all predictions and all analyses are simulated in a computer. Wet lab means we need to, in a real laboratory, in a room, use test tubes, use petri dishes, and perform operations on mouse cells and so on. So this is the difference between what can be achieved inside the computer and outside the computer.

When I give AI a problem, it is impossible for me to know the answer. GPT gives me a drug's molecular formula and says this can treat cancer. Claude gives me another molecular formula, and I have to do experiments, do this experiment. So before doing this experiment, it is not just about giving me a molecular formula; it also needs a specific experimental plan: how can this molecular formula be synthesized? How should this small molecule be stored? On what cycle should I administer it to mice? These very detailed things are what current AI models do very poorly. It can generate a molecular formula because there is a lot of such data in the world, but how is this molecular formula synthesized? How is this drug given to mice, and so on? They do all of these poorly. These are actually more important.

Hongjun: For the latter, do you think, looking at current large models, roughly what proportion of experiment design has been solved in designing an experiment? How is the completeness?

Wang Sheng: The completeness is probably only 30%, and many times human experts are needed to revise it. In the field of experiment design, specialized knowledge is needed at an extremely deep level. A professor in this field may not know either; it requires technical personnel who have done many experiments specifically in this direction to be able to design an efficient experiment with a high success rate and that is particularly feasible.

Also, every laboratory has different equipment or devices. An AI model like Astra cannot design experiments in a targeted way based on these devices and equipment. What I think it lacks is the ability to utilize, coordinate, and use resources in the real world. This is also what we now want to improve in the model.

Wang Sheng: I think it definitely needs to be equipped with a laboratory, because current general models are already very powerful, and with a laboratory it can have its own verification and accumulate its own data. There is another bottleneck in AI doing this kind of drug discovery now, which is that its verification cycle is very long. I design a molecule, and then I use mice to do an experiment on this molecule, which inevitably takes several weeks, because mice have metabolism and will react in some way to the drug. In these several weeks, the model may already have evolved to the next generation; AI models are evolving faster than my real experiments.

So I think this kind of frontier AI for Science research lab needs to have a wet lab, and it needs to renovate these laboratories so that these wet lab experiments can better cooperate with AI. For example, as soon as we start, we prepare all the cells and everything, all the preconditions are ready, and after there is an AI prediction, those small molecules are immediately put in, so the closed loop can run faster, rather than being separate.

Anthropic's molecular biology laboratory Image source: Anthropic YouTube

Wang Sheng: Generally we first do experiments on cells and mice, which can be completed in a few weeks. After that we will do a batch of experiments, for example on primates such as monkeys, which takes at least a 4-month cycle, and sometimes we do it twice, which is 8 months. As an AI pharmaceutical company, we may not necessarily advance to human experiments; we will directly license it to large pharmaceutical companies, and this licensing fee is also very considerable, generally more than 10 million US dollars, and can even be higher. Large pharmaceutical companies have a patient network and can better collect patients and conduct targeted human experiments. So we generally go as far as the end of monkey experiments, and the cycle may be 6 to 12 months.

Hongjun: So to verify a drug, the time it takes is far longer than for AI to design a drug. We say the whole of AI is already very smart, and it develops very fast in the field of mathematics because mathematics can be verified in many steps and does not require experiments. Then in biology, when AI really designs a new molecule or a new drug for you, how do you determine that its design is correct?

Wang Sheng: I think this is the biggest difficulty, and this is also why AI can make great achievements in mathematics and tackle the Millennium Prize problems. But in biomedicine, there is actually still no drug that can be said to be completely developed by AI, and the fundamental reason is that feedback from the real world is too slow, and this is limited by the laws of nature; a mouse or a monkey needs that much time.

In the previous AlphaFold era, many AI drug discovery companies sprang up like bamboo shoots after rain. At that time it was still before large models, but most of those companies did not succeed later because they still mainly focused on AI design and drug discovery. Even if they could shorten the cycle of AI designing a molecular formula from half a year to a few days or a few hours, they would still be constrained by the later real mouse and human experiments.

So I think this is a core problem. If this problem can be solved, the explosion of AI in biomedicine and pharmaceuticals will be as huge as AI solving the Millennium mathematics problems. I think this problem cannot be fundamentally solved, but we can accelerate feedback in the real world, thereby indirectly helping us solve and improve the effectiveness of AI applications in this area.

Hongjun: I heard that AlphaFold 3 open-sourced all of its research, including its data. Its open sourcing also brought a wave of biopharmaceutical companies, including companies doing vertical models and tools. Because after it was open-sourced, it was equivalent to your industry, similar to the robotics industry after Pi was open-sourced; many underlying infrastructures are already ready.

Wang Sheng: I think its open sourcing is very important. After these tools were open-sourced, the best help to the community is that everyone pays more attention to accumulating data. Although it has been open-sourced, some vertical scientific AI companies can use the data they have accumulated to maintain their moats. In fact, these data are extremely precious; a single data point costs tens of thousands or hundreds of thousands of dollars, because they all involve experiments on real mice and humans. Not only are the experiment costs high, but the experiment cycles are also long. So instead, after open sourcing, these vertical scientific companies will have more advantages. So this is its contribution to public resources.

Hongjun: Looking at it this way, is it possible that many data companies will also emerge in the biopharmaceutical field in the future? Anyway, your model is responsible for giving ideas, and I will set up a lab specifically responsible for validation. This may also be a very good business model.

Wang Sheng: I think this is a very good business model. The current prices of AI data, for example, buying an Olympiad math problem costs thousands of dollars, and a Terminal-Bench task also costs thousands of dollars. The newly released Terminal-Bench Science, the problems on it will also be high-value problems. I think these problems are still limited to operations within computers. In the future, if there is real real-world biological data, that value will be very high.

There is also another very high-value direction, which is building an interface that can help AI conveniently conduct real biological experiments. There is now a company called Adaptyv Bio, which specifically partners with Anthropic. Anthropic generates an AI prediction, and in the form of MCP it immediately starts doing experiments. It just raised $40 million.

I think there will be more and more such companies. In the future, every wet lab company will use this MCP and API form to connect directly to AI large models, thereby directly providing services to AI large models, and in this way a lot of human communication costs in the middle will be removed. So this brings us back to the question I just mentioned: the most important thing is that AI itself knows what experiment to do and how to design the experiment, and wet lab companies can cooperate with it.

Hongjun: Now, like OpenAI and Anthropic, which of the frontier model research labs we see have equipped themselves with their own wet labs?

Wang Sheng: Anthropic and Adaptyv Bio are clearly cooperating. It recently launched a competition, inviting everyone to work on a protein design problem defined by Anthropic. It will sponsor Adaptyv Bio to do wet lab validation for everyone. So the competition recently defined by Anthropic also says that wet lab validation will be the final standard.

I have also heard a lot of news that Anthropic itself is building a wet lab team, including that some of the people it hires also have wet lab backgrounds. So I think this is a future trend. If frontier labs really want to make high-value discoveries, it is inevitable that they will need their own wet labs.

Hongjun: In terms of the impact of models on pharmaceuticals as a whole, for you, what are the several major milestones? Which improvements in foundational model capabilities will cause this industry to suddenly have a qualitative leap or improvement?

Wang Sheng: Our current improvement almost still depends on improvements in coding capability. For these coding tasks in biomedicine, although those benchmarks have not yet reached full marks, my personal understanding is that they can almost be considered solved, because it is already better than what humans can do. So improvements in coding capability will solve the improvements in capabilities on the dry lab side of biopharmaceuticals.

There is also another issue: biomedical data is especially large and especially complex. Each person's data can reach two terabytes. So if future models still need improvement, it is in this area: efficient processing and understanding of this kind of two-terabyte data. This is a very important field and also a very big opportunity.

Hongjun: You just mentioned that you think AI has already surpassed you in some capabilities, because you are already considered a top scientist among humans and also do academic research. I can understand that it is no longer surpassing average intelligence, but the very top level of intelligence. In this process, what problems do you think it will bring to frontier scientists like you in the future? For example, it will be hard for you to validate what it does. Will such situations exist?

Wang Sheng: Validation also depends on experimental validation. If a problem has already been defined very well, with a clear test set and clear inputs, and you know what kind of problem is to be solved, then AI's ability to solve this problem will be stronger than mine. Then what human scientists can do more is define which problems AI should solve. So I think the future trend of AI in the biomedical field is no longer solving problems and designing models, but more about designing benchmarks, designing evaluation methods, and defining some important problems to solve.

Also, biomedicine is related to pharmaceuticals or treating patients, directly dealing with people, so many times a scientist is still needed to participate, just as even if an airplane can fly itself, we still want a pilot on board. AI can look by itself, treat by itself, and issue diagnoses, but I also want a doctor to really look at my report with their own eyes.

So in major decisions in scientific discovery that immediately involve a lot of money and a lot of time, a human scientist is still needed to jointly gatekeep and jointly judge the value of the problem. If it is purely about solving problems, I think AI is already stronger than the strongest human scientists.

Hongjun: You just mentioned that the definition of the current benchmark has problems. Then how do you think its benchmark should be defined in the future? In what direction will it evolve?

Wang Sheng: I think in the future, first of all, we need to design biomedical problems that have no upper limit, that cannot be maxed out. For example, the most advanced drugs today can kill 30% of cancer cells. Can AI make better drugs that can kill 40% of cancer cells? In this way, AI can keep designing, and there can be a way to do this task with no upper limit.

What we want to design in the future is still this kind of relatively grand, hard-to-cap task. For example, how to extend human lifespan. This kind of task with no upper limit can let AI keep solving it, and we can keep accumulating data, accumulating more data for a specific problem.

Hongjun: I saw that Claude recently had a new result, which is that they discovered a new enzyme system, ART. It has a gene-editing tool similar to CRISPR-Cas9, but whether it has ultimately been validated is still not very certain. In the biomedical field, it has already begun to have some breakthroughs. What are the signs of these breakthroughs we are seeing?

Wang Sheng: What we observe is that this breakthrough is almost directly proportional to the cycle of biomedical experiments. The shorter the cycle, the faster the feedback. Everyone has actually accumulated a lot of experimental data over the past few decades. In fields where experiments are done faster, the models can be made better. For example, the competition Anthropic did this time was also about enzyme design, which is a field where experiments are especially fast. In the future, fields like protein enzymes where rapid experiments can be done, I think, will all have very fast iteration. But if, for example, we want to work on aging, or study a rare disease, this kind of long-cycle problem in a specific field, then I think the progress of models will still be very slow.

Hongjun: What changes in which fields do enzymes and proteins bring?

Wang Sheng: Theoretically, proteins are omnipotent. Weight loss, cancer treatment, all diseases—theoretically, proteins can guide us in making drugs to treat them through some methods. Enzymes are a kind of catalyst, more about processing and accelerating various industrial chemical processes. Their commercial value is very large, and they can be deployed quickly. As for proteins, if I still want to get involved in drugs, ultimately human trials are unavoidable, and there is still a relatively long cycle.

Hongjun: I have also been talking with many model researchers. Many researchers at frontier model labs say that in fact many ordinary people may not even have distillation value anymore. This sounds very sad. Is the entire training signal now also shifting toward expert data and verifiable environments? Do everyone agree with this view?

Xu Huazhe: In the robotics field, this is obviously not the case. In robotics, everyone is an expert in life. You are good at cooking, I am good at folding clothes, he is good at walking, he is good at dancing. Of course, going further, it will also go through such a stage. Maybe in a year or two, one must be a dance expert, or must be a chef like Gordon Ramsay, to be worth being distilled.

Wang Sheng: If knowledge is distilled, then indeed AI's knowledge is now more comprehensive, and a single scientist certainly cannot compare. But there are some scientific discoveries, some long-term investments that still need scientists or engineers there to support and endorse them, so in this respect AI cannot replace them.

At the same time, there is also a reverse distillation problem. I am also distilling AI every day. Especially those of us doing the interdisciplinary field of AI for Science, there are many things I do not know, and I can quickly learn from AI. After learning, my speed of acquiring knowledge is also fast. Sometimes, with some new ideas and new creativity, I can produce better scientific research results. So I think this is a process of mutual distillation.

On the other hand, from the perspective of our specific field of medicine, the role of humans will always be very large. Humans not only solve problems, humans can also define problems. For example, there are some new diseases, or people want to live to 200, live to 500. These problems are endless. Then everyone will have their own different physical conditions, continuously providing AI with new problems, so ordinary people will absolutely not be replaced by AI.

Zhu Zheqing: From the perspective of distillation, because our current intelligence, or model capability, is still in very large models, which may have rote-memorized a lot and a lot of knowledge, and some know-how is also forcibly added during training. For example, when you are optimizing some benchmarks, or advancing some fixed deployment scenarios, you will be in a state of a lot of overfitting. In this case, actually the fields we currently overfit may not even be enough. There are many, many fields worth overfitting.

We all say now that models may have reached the level of general intelligence, but from the scenarios we have currently overfit, they may not have reached such a general state. Later, if we really want to achieve generality, we may still need to jump out of distilling each scenario and enter a state of simulating a person learning from scratch. From this perspective, the distillation capability we actually need is not about whether I can do this thing well, but about distilling how humans learn, and how to learn from scratch even faster than humans.

Hongjun: Today is very rare. We have invited three entrepreneurs and scholars from three different fields who I think are doing well. Would everyone like to say what is the one issue in your field that you care about and pay most attention to right now?

Zhu Zheqing: The issue we are most focused on right now is actually that, on current hardware, storage worldwide is very limited, memory is very expensive, and VRAM is very expensive. Many people also worry: are we putting all the data in the cloud? If in the future we believe that all computation and compute power will be spread out, then everyone will have their own compute, and every company will have its own compute. Compute will no longer be concentrated only in data centers. Can we imagine that everyone can have their own compute and run their own models? These models do not necessarily have to be the largest. Can we use these models to solve 80 to 90 percent of the problems in life? To solve the problem that AI currently cannot be popularized in a completely unlimited way?

I actually still agree that future models need to develop toward being larger and stronger. For example, in AI for Science, I may need a model with very high depth. But in normal settings, such as enterprise work and personal life, you may need a model with very high breadth, and this model does not need to be very large. So this is a question of combination. If the right combination is achieved, it can actually solve the current bottlenecks in hardware demand and power demand.

Xu Huazhe: From the robotics perspective, what we care about most is certainly zero-shot generalization capability. Previous robots actually had two paths: one is deployment, and the other is generalization. These two paths are actually somewhat orthogonal. For deployment, it is a bit like AI empowering the previous generation of robots. For a robotic arm in a factory, it may originally have tightened screws in an open-loop way; open-loop means I do not take feedback. Now that I have AI, I can tighten screws in a closed-loop way, or load goods in a closed-loop way. But from my perspective, this is still somewhat against the trend of the times. You still need five PhD-level people to sleep in the factory for three months to solve all these issues, such as the robot's stability, speed, and whether the end effector gripper needs to be customized.

But today when we talk about Astra, I think ultimately we still hope for a general-purpose robot. It can achieve rapid deployment zero-shot. This deployment should be done by placing five arbitrary people who do not need to be very strong there. Such zero-shot generalization will certainly have redundancy. Is such redundancy acceptable to us? I think it is acceptable. In this case, whether there can be a sufficiently good and fast zero-shot generalization model is still the main pursuit of robotics.

Hongjun: How far do you think robots are from that step?

Xu Huazhe: I think early 2028 will be a time point when general-purpose robots begin to become truly practical. Starting now, perhaps 18 months later, people can use language at home to make a robot do 90 percent of the things at home.

Hongjun: This prediction is quite aggressive.

Xu Huazhe: If you have played with Astra, and if you may be able to play with our model next month, you should feel that this prediction is still relatively reasonable.

Wang Sheng: For AI for Science, if there is really to be a breakthrough, the most important thing is AI's understanding of the real world, the use and allocation of real-world resources, and so on. Just as when we study science, the essence is to study the real world: physics studies the universe, biology studies life, and chemistry studies the microscopic world.

But this does not mean that I design a robot to replace humans and automatically conduct experiments. Rather, it means that AI knows: based on what resources are currently available in my laboratory, I should have this robot, or this person, do certain things. For example, in the future when I prompt AI, I will not say to it, "Design me a drug that can treat cancer," but rather, "I give you 100,000 US dollars; design me a drug, and I have these experiments that can be done," such as a mouse experiment costing 2,000 US dollars, and what kind of cycle a human trial might require.

AI can truly plan such super-complex tasks in the real world in this way, and these are rarely seen in its corpus. This may require a new paradigm. Just as now we all have humans use AI, and humans study how to write good prompts to direct AI well, how to write good prompts, and how to build good agents; in the future, it may be AI that needs to write clear, executable instructions. It will be AI that prompts the real world: very efficiently, very clearly, and without ambiguity. Humans or robots can use these instructions to utilize resources in the real world and complete these things efficiently.

Just as we used to say that judging whether AI can follow human instructions, in the future we may need instruction generation: whether it can generate better instructions to help us mobilize real-world resources. We may even make some modifications to the real world: some future experimental equipment and instruments will not be for humans to look at or for humans to use, but for AI to use. For example, now when taking X-rays or CT scans, we still need to synthesize images so that human doctors can look at them; in the future, perhaps the underlying waveforms or signals can be taken directly and shown directly to AI, skipping the step of generating images.

So this is what I think is most important: AI can better step into the real world, whether through robots or through a real human, but it must know how to use real-world resources to solve such extremely difficult problems.