AI Trends

Yang Ruigang, Founder of Niuwa Robotics: The Data Pitfalls Autonomous Driving Fell Into Will Be Repeated by Robots | Physical AI 50

English text is being prepared.Updated Nov 18 21:27

AI summary

Niuwa Technology founder Yang Ruigang has shifted from autonomous driving to entrepreneurship in embodied intelligence for community last-mile delivery. He assesses that generalization or cost barriers are too high for home and factory scenarios, whereas community delivery can reuse localization and navigation technologies while gradually incorporating interaction. Niuwa proposes "Embodied Action Intelligence," building a system around the World Travel Model (WTM) for data generation, physical simulation, and model training.

Body · AI translation

After more than a decade in autonomous driving, Yang Ruigang decided to shift his next entrepreneurial venture from highways to residential communities.

Before this, he had studied 3D vision at U.S. universities, joined Baidu Research Institute to lead the Robotics and Autonomous Driving Laboratory, and served as CTO of Inceptio Technology, promoting the pre-installation mass production of autonomous heavy trucks. From researching algorithms to getting dozens-of-ton trucks truly driving on highways, he witnessed the process of autonomous driving moving from technological exploration to commercialization.

In 2024, Yang Ruigang began preparing for his second startup. At that time, humanoid robots had become the most watched direction in the embodied intelligence industry. He also explored related entrepreneurial opportunities but ultimately chose a scenario that seemed less "sexy": community last-mile delivery.

He judged that while home robots have a large market space, every household's environment is different, making generalization extremely difficult; factory robots must meet strict production cycle times and cost requirements. In contrast, community delivery can reuse positioning, navigation, and obstacle avoidance technologies accumulated from autonomous driving, and allow robots to begin interacting with the real physical environment through tasks such as opening doors, taking elevators, and delivering packages.

This was the starting point for Yang Ruigang founding Niuwa Technology. He hopes to first let robots solve mobility and passage issues, then gradually move into more complex tasks. To this end, Niuwa proposed "Embodied Mobility Intelligence," building a system for data generation, physical simulation, and model training around the World Travel Model (WTM).

From witnessing autonomous trucks driving on open roads to robots handling various complex tasks between buildings, Yang Ruigang has always been challenging technically unsolved problems. Yet he says the real opportunity lies precisely in those technical problems that have not yet been solved.

"We are not afraid of difficulty." Re-understanding the value of technology from the laboratory to mass production

Yang Ruigang: Actually, I didn't start out doing computer vision; I did computer graphics.

I was in the CAD and CG lab when I was at Tsinghua University. Later, I went to the University of North Carolina at Chapel Hill for my Ph.D. because it was very famous in the field of graphics.

When I first arrived, I mainly worked on graphic rendering and display, and later participated in a project called Telepresence. It aimed to solve the problem of whether multiple cameras could be used to collect a person's 3D information in real-time and render it from different perspectives, allowing people thousands of miles apart to hold meetings face-to-face.

As we worked on it, we found that rendering and display were gradually no longer the hardest problems; the real challenge was how to reconstruct 3D information in real-time.

So my doctoral research shifted to 3D vision, studying how to recover 3D structures from images, point clouds, or depth information. Later, I began researching semantic information, not just knowing the 3D shape of objects, but also what they are.

I entered this field in 1998, long before the rise of deep learning. UNC was very strong in graphics at that time, and I was among the early batch of people submitting papers to CVPR.

But I didn't feel like I was sitting on a cold bench, because there were still many unsolved problems in 3D vision itself.

Yang Ruigang: The earliest applications of deep learning in computer vision were mainly concentrated on tasks like image classification, detection, and segmentation. The widespread adoption of deep learning in 3D vision actually came much later.

My earliest attempt was to introduce constraints from traditional 3D geometry into deep learning frameworks. At that time, I also collaborated with Su Hao and others to study how to combine 3D geometry with deep learning.

Later, I worked on directions such as generating new views, diffusion models, and video generation control.

For researchers, whenever new technologies emerge, one must relearn. It's just that the learning frequency is getting faster now; before you fully understand the previous things, new methods may already have emerged to replace them.

I believe scientists must possess the ability to learn from scratch.

Yang Ruigang: We conducted large-scale street scene 3D reconstruction for simulation training. The U.S. placed great emphasis on using simulation systems, hoping to understand streets and terrain through 3D reconstruction before entering real environments. We did this even earlier than Google Street View.

We also researched how to convert very dense point clouds into simplified polygon models while retaining high-quality textures. Later, Apple Maps' 3D maps adopted similar technical approaches.

Commercialization attempts were made as well. I received SBIR project funding twice in the U.S. These projects provide hundreds of thousands of dollars with the purpose of pushing paper results toward product conversion.

But at that time, it was basically informal. If interested, we continued working; if not, we might look for jobs.

At that time, we lacked strong business awareness and didn't have anyone particularly skilled in commercialization to drive us, so nothing really took off.

New Smart Driving Physical AI: You had already done years of research at U.S. universities and attempted technology commercialization. Why did you decide to join Baidu in 2016? Who invited you?

Yang Ruigang: Lin Yuanqing invited me, Andrew Ng was responsible for the interview, and finally, I had an online exchange with Robin Li.

Lin Yuanqing and Andrew Ng left successively later. For about two and a half years, I mainly worked under the leadership of Wang Haifeng.

In 2016, Baidu Research Institute had about five or six scientists like me. Each was responsible for a relatively independent laboratory with significant autonomy over personnel and technical directions. The operational style was actually quite similar to university labs. Later, I was responsible for the Robotics and Autonomous Driving Laboratory.

New Smart Driving Physical AI: During your time at Baidu, you promoted the ApolloScape dataset. Why was this project undertaken at that time? How much resources were invested?

Yang Ruigang: This project owes its existence to Lu Qi.

The core team for ApolloScape consisted of about 10 people, focusing on work for about half a year. Participants included colleagues such as Huang Xinyu, Geng Qichuan, and Zhou Dingfu, who were respectively responsible for data collection, cleaning, tool development, etc.

We also received support from other teams within Baidu. For example, that set of high-precision data collection vehicles, which used high-precision LiDAR, as well as navigation-grade IMUs and GPS, and we added some extra sensors.

ApolloScape's data was 100% collected from real roads, not generated by simulation. However, later the company's business focus changed, and the dataset did not receive as much continuous construction and maintenance investment as originally intended.

After Lucian Ye joined Baidu, he placed great emphasis on building a technical brand. He reasoned that since the autonomous driving field already had well-known datasets like KITTI, Baidu should have its own data brand, and this task was assigned to me.

The core team for ApolloScape consisted of about 10 people and worked intensively for around half a year. Participants included colleagues such as Huang Xinyu, Geng Qichuan, and Zhou Dingfu, who were respectively responsible for data collection, cleaning, tool development, etc.

We also received support from other teams within Baidu. For example, that set of high-precision data collection vehicles, which used high-precision LiDAR, as well as navigation-grade IMUs and GPS, and we added some extra sensors.

ApolloScape's data was 100% collected from real roads, not generated by simulation. However, later the company's business focus changed, and the dataset did not receive as much continuous construction and maintenance investment as originally intended.

Yang Ruigang: What impressed me most was the business model proposed by Winchell, called "dual-driver to single-driver."

Many long-haul freight operations in China originally required two drivers taking turns driving. If a sufficiently reliable driver-assistance system could be developed, allowing one driver to transition into a safety officer, there would be an opportunity to reduce two drivers to one. This would not only improve transportation costs but also accumulate data continuously during actual operations, gradually evolving toward L4 autonomy.

I believe this is a progressive route. If one attempts to build L4 directly from the start, mass production and large-scale deployment would be extremely difficult. Especially for heavy trucks operating on highways, the issue of road rights is even more complex.

In comparison, first developing commercializable driver-assistance systems and then gradually increasing the level of automation is a more realistic choice.

Yang Ruigang: When I joined, my main responsibility was to establish a complete R&D system for autonomous driving technology.

At that time, the company had R&D centers in both China and the US. The initial approach was somewhat similar to "US R&D, China implementation." But after I entered, I explicitly stated that this model was hard to sustain in the long term. Autonomous driving requires extensive real-world testing. Since the product would ultimately be implemented in China, the R&D team needed to be as close to real scenarios as possible.

Later, the Chinese R&D team grew from about 10 people to around 200, while the US team shrank from approximately 50 to about 10, and further reduced to single digits.

I have always believed that research can adopt a relatively loose cross-border collaboration model, but for true mass-produced products, R&D, testing, and delivery must be closely integrated. Beyond on-site testing, cross-border collaboration also involves practical constraints such as chip supply and legal regulations.

Yang Ruigang: I often say: We are "dancing with handcuffs on."

Because for mass production, many hardware conditions are determined in advance.

For instance, chip computing power and sensor configurations are already fixed. You cannot change to a better card just because you feel the computing power is insufficient, as one might do in research.

In mass production, a few serious failures can prevent the product from leaving the factory. Making a product is like a wooden barrel; the shortest plank must be long enough.

Heavy trucks are a prime example. The difference between empty load and full load is significant, and the dynamic relationship between the tractor and trailer is complex. We also spent considerable time on fuel efficiency. Commercial vehicles are production tools; customers calculate costs and efficiency, and these issues require technological answers.

Yang Ruigang: Yes. I have accompanied vehicles many times, communicated with drivers, and met with vehicle manufacturers and heads of express logistics companies.

I enjoy this process because it allows me to understand customer needs from completely different perspectives.

As a CTO, one cannot only care about technology; one must also know who the customers are, why they buy this product, and what problems they truly want to solve. The biggest taboo for technical leaders is working behind closed doors. Embodied AI Implementation: Find Match First, Then Discuss Generality

Yang Ruigang: When I seriously considered starting a new venture again in 2024, our earliest name idea was Numa, which can be translated to "Mu Niu" (Wooden Ox) corresponding to the Wooden Ox and Flowing Horse. I felt the English pronunciation was simple, easy for foreigners to read, and sounded somewhat like Puma; I had even registered the domain name.

Later, younger colleagues, especially HR, advised me strongly against using this name because in Chinese it sounds like "Niu Ma" (beasts of burden/slaves). It might be okay for the founder to self-deprecate, but it seemed inappropriate for employees to constantly talk about working at a "Niu Ma company."

So we continued adjusting the original name, eventually arriving at "Niu Wa." "Niu" means connection or bond, and "Wa" comes from Nuwa, the creator of life. We hope to create a new form of robot, or even a new species.

Yang Ruigang: I always ask where the product-market fit lies. There is still a long way to go from technological breakthroughs to large-scale applications.

Both homes and factories have their own difficulties.

Homes appear relatively enclosed, but every home environment is different. Item placement, furniture structure, and user needs vary greatly, requiring very high generalization capabilities.

The main issues in factories are cycle time and efficiency. A robot might be able to perform a certain action, but if its speed is far slower than manual labor or its cost is several times higher, customers will find it hard to accept.

In comparison, last-mile delivery does not have such high requirements for fine manipulation and action cycle times.

In community delivery, part of the problem is familiar to us, such as positioning, navigation, and obstacle avoidance. Moving forward, the robot needs to swipe access cards, open doors, take elevators, and hand packages to users, interacting with the physical environment. This is an interesting intersection between autonomous driving and embodied intelligence.

This scenario has another advantage: you can choose which communities to start with. There is no need to solve all problems at once; instead, find suitable communities, get the system running, and then gradually expand the scope of application.

Yang Ruigang: We initially thought that old residential communities without elevators and with the most difficult delivery conditions would be the ones in greatest need of robots. But after talking to customers more, we found that the market taught us a lesson.

They suggested starting with high-end communities. On one hand, these communities may have a stronger willingness to pay; on the other hand, existing technology is easier to meet their environmental requirements. This feedback directly affects the product form. Originally, we first considered climbing stairs, but now we look more seriously at which scenarios can be met with wheeled chassis.

We are clearer now about which features customers truly need and which are temporarily unimportant, and the mass production plan has therefore been advanced.

New Smart Driving Physical AI: Have you ever fully followed the workflow of a courier or food delivery rider? What is the hardest 100 meters?

Yang Ruigang: We have followed them many times and talked to many couriers and food delivery riders. The thing they are least willing to do is go upstairs.

For couriers, it is best to deliver packages directly to the station, where sorting, shelving, and distribution are handled by others.

Food delivery riders are the same. For example, in Huaqiangbei, Shenzhen, some commercial buildings have people below who help riders deliver food upstairs, some of whom are retirees. During lunch and dinner peak hours, they wait downstairs, and riders pay about 1 yuan for them to take the food upstairs.

Because riders calculate costs, assuming an order takes 20 minutes, with the last 10 minutes spent waiting for the elevator and going upstairs. A rider might earn six or seven yuan for this order, and they would rather spend 1 yuan to have someone else complete the last leg, using those 10 minutes to pick up the next order. If they can earn three more yuan during this time, minus the one yuan paid out, they still earn two more yuan. They definitely calculate this account better than we do.

SF Express even has a vivid internal term called the "Upstairs Machine Project."

Yang Ruigang: I think the business models are quite different.

The demand for delivery in hotels is limited, and the number of tasks a robot can perform daily is also limited. If the utilization rate is not high enough, the economic equation is hard to justify. Moreover, the technical threshold for hotel delivery is relatively low, mainly involving SLAM and path planning. With more participants, it easily falls into price competition.

But the number of express and food delivery orders generated in a residential community every day far exceeds the delivery needs of a single hotel. The volume of last-mile delivery tasks is larger, and the potential usage frequency is higher.

Of course, this presupposes that the technology can truly solve the problem.

New Smart Driving Physical AI: When robots enter communities, what other Corner Cases will they encounter that were not anticipated in the lab?

Yang Ruigang: There are too many such situations. We encountered an interesting scenario.

A robot encountered a person on a very narrow road. That person was very polite, waving continuously to signal the robot to go first. But the robot did not understand this gesture and kept stopping there, while the person did not move, continuing to wave. As a result, the person and the robot were stuck in a standoff.

Another overseas customer asked us if the robot could get back up and continue executing tasks if kicked over. Especially for quadruped robots carrying a large package on top, if the entire unit is flipped over like a turtle with its four legs in the air, it is actually difficult to restore its normal posture on its own.

Previously, colleagues working on mall cleaning and security robots encountered a situation where four or five people lifted a robot weighing over 100 kilograms to another corner. Because that robot was based on SLAM navigation, losing its initialized position after being suddenly moved, it could only stop there.

What we focus on more is safety issues. For example, after a sudden power outage in the mechanical structure, the robot might automatically sink. If a child places their hand near the joints at this time, there is a risk of pinching or crushing injuries.

Many problems can only be discovered on-site after robots truly enter human living environments.

Yang Ruigang: ROI must still be considered.

For example, opening fire doors is technically feasible, but some fire doors are heavy and require large torque, meaning heavier and stronger robotic arms are needed. After the robotic arm becomes heavier, the main body must also be reinforced, but some community elevators are very small. If the robot is made too large, it will struggle to turn around once inside.

So the question is how much cost needs to be added to open this door, and what new limitations it will bring.

Sometimes, simply installing a door opener on the door, costing around 500 to 1,000 yuan, can solve the problem. In comparison, modifying elevators involves more laws, regulations, and approval requirements, making it much more complex.

Currently, customers' clear requirements are: first, do not modify the elevator's control system; second, the robot must be able to pass through access control. I feel that at this stage, there is no need to require robots to solve all Corner Cases before commercialization.

There is another big difference between last-mile delivery and Robotaxi: Robotaxi must be able to operate in a large continuous area for the commercial service to be viable; delivery robots can first select suitable communities, truly run through one community, generate revenue, and then gradually expand the scale.

Yang Ruigang: Light mapping still uses maps. We use lightweight topological maps to provide basic spatial relationships, do not rely on high-precision maps, and combine real-time perception to advance the route.

This direction aims to reduce the burden of creating and maintaining high-precision maps. However, what initialization, facility configuration, and testing are needed for new sites depend on the specific environment. Judging whether it has value ultimately comes down to how long it takes to prepare a new site and how many adjustments are needed after on-site changes. Mo's true barrier is the system closed-loop.

"New Smart Driving Physical AI": Many embodied intelligence companies are currently working on end-to-end models, world models, or VLA. What is special about Niuwa's model architecture?

Yang Ruigang: Frankly speaking, I feel that the differences in model architecture among various companies at this stage may not be as significant as imagined. The core capabilities that Niuwa has already verified mainly lie in the simulator, data generation, and data augmentation.

What truly affects performance is how much data you have, what situations these data cover, and whether you know which data to use to train for the desired capabilities. There is a lot of experience involved; obtaining a model architecture does not mean you can accomplish the task.

When I was working on ApolloScape at Baidu, I realized that real-world collection could hardly cover all long-tail scenarios.

Yang Ruigang: For example, to collect fire trucks, we would station our collection vehicles near fire stations and follow them once they came out. But you cannot cover all situations with this method. As time goes on, the proportion of truly valuable information in the collected data decreases.

So we considered synthetic data very early on. Real data tells you what happens on site, and then you need to consider whether you can increase the complexity and diversity of the data. This understanding has persisted to the present.

"New Smart Driving Physical AI": What are the respective relationships between WorldWeaver, SimWeaver, and WTM in Niuwa Technology's core technical architecture?

Yang Ruigang: WorldWeaver is responsible for organizing real-scenario data, 3D assets, and generated data, while SimWeaver provides physical simulation capabilities. This data and simulation system serves the training and evaluation of WTM.

We hope that problems discovered on-site can enter a repeatable testing process. For instance, if an interaction fails, and we can reproduce the corresponding scenario and conditions in simulation, we can repeatedly compare different handling methods before verifying them on the actual machine. In this way, data becomes connected to product issues.

Yang Ruigang: We have performed migration between quadruped robots of different brands. Since the two bodies are relatively similar, it was easier. Now we are also migrating the system to wheeled chassis.

Quadrupeds and the wheeled chassis we selected have different modes of movement. Some lateral movements possible for quadrupeds require reprocessing when applied to this type of wheeled chassis, necessitating adjustments to trajectory tracking, control, and smoothing. The complexity of underlying control also differs.

Currently, the entire system can basically run on the wheeled chassis.

Yang Ruigang: There is no clear commercial demand yet. At this stage, both quadrupeds and wheeled platforms have customer demands, so we still hope to prioritize doing things that can generate practical value as soon as possible.

We look at which tasks in human living environments require robots to move and execute. The capabilities extracted from community delivery can be used for other tasks in the future, but they need to be verified according to specific scenarios.

Yang Ruigang: Definitely needed; we are currently customizing our own wheeled robots.

Because community delivery places very specific requirements on the robot body, you cannot simply take any off-the-shelf wheeled robot to complete the task.

For example, a customer explicitly stated that the robot must be able to climb slopes of 8 to 10 degrees and cross curbs of 5 centimeters. 5 centimeters is actually not low.

There are also size requirements. Although some residential building lobbies have double doors, usually only one side is opened, so the actual passage width might be only 50-60 centimeters. If the robot is too wide, it won't fit.

However, from the perspective of delivery efficiency, we want it to carry more items at once. The larger the load capacity, the taller and heavier the robot becomes. But if it is made too tall, it tends to become unstable or even tip over when going uphill. Therefore, body design is essentially a continuous trade-off process.

Yang Ruigang: I don't think they are contradictory.

First, the market is large enough; robot companies like Deep Robotics cannot rely solely on us as their only customer.

Second, if partners are willing to use our "brain," we are also very willing to collaborate with them to design robots better suited for specific application scenarios.

Many robot manufacturers already accept OEM or joint development cooperation models. We can be responsible for the brain and application capabilities, while they handle body manufacturing. Both parties can jointly develop products, forming a complementary relationship.

English text is being prepared.

We do not plan to hold a large number of robots ourselves for large-scale operations. Operations require strong professionalism and can be cooperated with enterprises skilled in this area. I hope Niuwa's core capabilities remain focused on robot intelligent systems and hardware-software development.

Yang Ruigang: At the end of September this year, we were invited by K2 to participate in LIVEX in Abu Dhabi, where we jointly displayed products with their subsidiary AutoGo. We are also advancing PoC cooperation with K2 in the UAE.

Next, we hope to work with local partners to clarify specific scenarios and tasks. We need to determine item by item: what roads will the robots traverse, which facilities will they use, and who will handle on-site problems. Attending exhibitions allows customers to see the products, but actual usage requires verification within specific scenarios.

We also want to see which capabilities can be directly reused in a different environment and which need adjustment. How much of the navigation, facility interaction, and task execution capabilities accumulated from community delivery in China can be reused overseas? After architectural and service processes change, what additional work needs to be added? These questions will help us judge which aspects are already product capabilities and which still depend on project-specific development.

We do not plan to conduct large-scale operations ourselves; instead, we prefer to focus our efforts on intelligent systems and software/hardware products. Therefore, it is necessary to clearly define the division of labor in overseas cooperation: at what stage does Niuwa's responsibility end, what support local partners will provide, and how to ensure that on-site issues are promptly fed back to R&D. We need to solidify specific projects first before considering expansion.

Yang Ruigang: We already have revenue, but it is mainly project-based and POC-stage income, without yet forming scaled revenue.

Yang Ruigang: Our team consists mainly of two parts. Currently, including full-time employees and interns, there are more than thirty people.

One part comprises young research talents, mostly students and research teams from Shanghai Jiao Tong University (SJTU). The other part includes individuals with mature engineering, product, and commercialization experience, many of whom are former colleagues of mine from Baidu and Winchell.

University resources have been very helpful to us. Now, SJTU can provide excellent students and research talents, while Niuwa can offer more computing power, data, and real-world scenarios.

Yang Ruigang: The first institution explicitly willing to invest was PNP.

We started contacting the financing market around November 2025 and received the first Term Sheet (TS) from PNP in December.

The communication at that time was quite fast. After chatting with their partners about twice, things became quite clear. They have three partners in total, one of whom is primarily based in the United States. I specifically visited him in January 2026, after which the process mainly involved due diligence and other procedures.

What they valued most was actually the completeness of the team. As for the specific direction, they felt that more exploration could be done, and it did not need to be completely determined from the start. Because at that time, we were indeed a very early-stage team, with basically everything still on PowerPoint slides.

After the seed round, we quickly completed another angel investment round of 50 million yuan.

The second round progressed even faster. The head of an investor was a former colleague of mine from Baidu who knew me well and understood our team's capabilities, so many things did not require repeated explanations. After one meeting, the other party even said that verbally, they could issue a TS right now.

I think this also shows that early-stage financing largely invests in people and teams.

New Smart Driving Physical AI: The original plan was mass production in 2028. Why has it been significantly accelerated now?

Yang Ruigang: There are mainly two reasons.

On one hand, it is competition. When we first started doing last-mile delivery, it felt like few people were working in this direction, but now several companies have begun to layout.

On the other hand, it is market demand. As we contacted more customers, we found that actual needs were more urgent than originally anticipated.

From receiving the TS in the previous round to deciding to accelerate the next round of financing, there was only about three months. But during these three months, we achieved some very clear technical milestones, and the feedback obtained commercially was much clearer than before.

We now have a clearer judgment on which functions customers truly need and which are temporarily unimportant. Therefore, the mass production plan has been advanced by at least half a year compared to the original schedule, and the financing rhythm has accordingly sped up.

Yang Ruigang: I believe cumulative financing must reach at least the hundred-million-yuan level to have sufficient funds to bring the product to a mass-production state that is truly meaningful and capable of sustainable, stable operation.

Doing POCs is relatively easy, but going from 10 units to 100, and then to 1,000, model training, data, and manufacturing will begin to consume significant capital. More importantly, truly enabling robots to operate stably in 100 communities is a completely different matter from making a few POC prototypes.

We are currently only operating in a few communities. There is at least an order-of-magnitude growth needed to reach 100 communities. The real challenge is ensuring long-term stable operation in a vast number of different real-world environments.

I often tell the team that if this task were not difficult, it would not fall to a team like ours to do it. There are many people outside who understand the industry, operations, and sales better than we do.

Our true opportunity lies precisely in the fact that some technical problems have not yet been solved. Because it is difficult, that is why people like us who work on technology are needed.

We are not afraid of difficulty.

Source

English text is being prepared.