Iluvatar CoreX's Tiangai 300 Benchmarked Against H200, Company Also Discloses Edge-Side Tongyang Roadmap

After reaching H200-level performance in real-world test scenarios, competition among domestic GPUs is shifting further toward system efficiency, software adaptation, and actual delivery.

At this week's Iluvatar CoreX media open day, the live comparison between the latest-generation Tianyao 300 and the H200 was the most intuitive performance demonstration. Combining GPUs with dedicated chips into a heterogeneous system is a new exploration of inference efficiency. The introduction of Agents and Skills into the software stack, meanwhile, attempts to solve the engineering challenges of customer deployment, adaptation, and optimization.

Iluvatar CoreX said that in a set of third-party tests completed this year, under a typical PD-disaggregated deployment, the Tianyao 300's normalized per-card end-to-end throughput reached 1.05 times that of the H200. This shows that domestic general-purpose GPUs already have the performance foundation to compete directly with the H200 in specific high-end inference workloads.

But from approaching in performance to customers purchasing, and then to sustained capacity expansion, system efficiency, software adaptation, and stable delivery still stand in between. When the hardware is competitive, how can domestic GPUs turn that capability into a reason for customers to keep using them? As inference gives rise to more dedicated chips, what is the unique value of general-purpose GPUs?

From the cloud-side Tianyao 300 to the device-side Tongyang series, Iluvatar CoreX is also expanding its product lineup, extending competition into computing-power demands of different forms.

Tianyao 300 vs. H200 live test: a slight performance lead

For a long time, peak computing power was an important metric by which domestic AI chips drew attention. But for end users, how many effective tokens a device actually produces per second is closer to the purchasing decision than the theoretical computing power on a spec sheet.

The third-party tests disclosed by Iluvatar CoreX cover four mainstream models and three types of scenarios. Of these, the 1.05 times figure refers to the overall normalized per-card end-to-end throughput under a typical PD-disaggregated deployment, reflecting system performance under specific models and deployment conditions.

In addition, according to data from Iluvatar CoreX's performance lab, in the 64K long-context scenario of GLM-5.3, while ensuring an equivalent user experience, the Tianyao 300's MoE throughput reached 1.15 times that of the H200. In Agent scenarios with long contexts of 8K to 128K, time to first token was reduced by about 10% to 30% compared with the H200.

The former concerns how many requests the system can handle, while the latter concerns how long users must wait after issuing a request; they correspond to different workloads and performance metrics.

The Tianyao 300's ability to achieve a performance lead in these scenarios hinges on architecture optimization for actual workloads.

Built on the Tianshu architecture, the Tianyao 300 has 144GB of per-card memory capacity and is optimized around computing stages such as Attention and MoE. Long contexts need to hold more states, which also increases the computational pressure of Attention. Although MoE controls the amount of computation by activating only some experts, it brings new problems such as small-scale matrix computation, expert assignment, and data exchange.

To this end, the Tianyao 300 lets Tensor Cores and Vector Cores process matrix operations in Attention and vector operations such as Softmax in parallel, while optimizing for small-scale matrices, low-precision computation, and communication coordination in MoE.

These targeted designs help reduce waiting between different computing stages and turn more hardware resources into actual output.

The tests and live demonstrations disclosed this time show the Tianyao 300's ability to enter the actual competitive range of high-end inference. For customers, the next thing to compare is how many requests the system can handle under the same response requirements, how it performs during sustained operation, and how much equipment, power, and operations-and-maintenance cost must be paid.

Inference demand gives rise to dedicated chips. Will the value of GPUs be diminished?

As inference demand grows rapidly, dedicated inference chips represented by LPUs are attracting much attention, and the hardware requirements of different computing stages are further diverging.

Prefill processes input and is usually more oriented toward compute-intensive tasks; Decode generates tokens step by step and is more susceptible to memory access, task startup, and synchronization overhead. Broken down further, Attention and FFN also differ in their demands for computation, bandwidth, and storage.

When a certain type of task is stable enough and large enough in scale, dedicated chips have an opportunity to achieve better performance or energy efficiency through targeted design. Having a single general-purpose GPU handle all stages does not necessarily always correspond to the lowest cost.

The AFD disaggregation scheme demonstrated by Iluvatar CoreX is a response to this division of labor.

On top of separating Prefill and Decode, it further splits Attention and FFN in the Decode stage: the Tianyao 300 handles the full Prefill computation process as well as Attention in Decode; a partner vendor's XPU handles FFN expert-network computation.

During operation, the GPU sends intermediate activation data to the XPU, and the XPU returns the results after computation to continue advancing model generation. Comparison results provided by Iluvatar CoreX show that, in the demonstrated scheme, Decode inference latency was reduced by 60%.

Heterogeneous division of labor gives different chips room to cooperate, while the generality of GPUs determines whether the system can continuously adapt to model changes.

Iluvatar CoreX Vice President Zou Xuan told Leiphone that dedicated hardware can achieve substantial performance gains in fixed workloads, but models and algorithms are still iterating rapidly, and the company insists on a general-purpose framework as the foundation, combined with dedicated acceleration capabilities.

After customers purchase equipment, they need to keep using it, yet model structures, operators, and precision requirements may keep changing. The generality of the GPGPU platform provides room for continued adaptation, reducing the risk of binding the device to a single compute path.

Reinforcement learning, scientific computing, and physical simulation also push compute demands beyond a single inference task. Iluvatar CoreX's demonstration of these applications likewise reflects the broader range of uses enabled by the GPGPU general-purpose architecture.

In the inference era, dedicated chips may take over part of the GPU's compute tasks, but GPUs can still participate in a wider range of computing by virtue of their generality. For domestic GPUs, finding a position in an efficient division of labor and consolidating that value through interconnect, scheduling, and software has become key to continuously attracting customers.

The finer the heterogeneous division of labor, the more work software must take on. Hardware provides the choices; software determines whether those choices can become deliverable services.

According to Iluvatar CoreX, it has served more than 380 customers across over 30 industries. Based on accumulated project experience, customer application migration usually takes less than an hour, while performance optimization may take from several days to a week.

Model tracking and migration efficiency are also important conditions affecting customer repurchases. To this end, Iluvatar CoreX's software stack has introduced Agents and Skills to reduce engineering costs in the deployment and optimization process.

Skills organize validated steps, judgment methods, and acceptance requirements; Agents call existing tools, read execution feedback, and then advance to the next step. Work that originally relied on engineers checking documentation, configuring environments, reading logs, and repeatedly trial-and-error can potentially be partially transformed into reusable processes.

PD-disaggregated deployment is a concrete example. It requires separately configuring Prefill and Decode instances, passing KV Cache, and connecting request routing and scheduling. A single component running normally does not mean the entire service chain is already usable.

According to the on-site introduction, in the complete model service deployment case demonstrated, the time from environment preparation to the model normally providing service was shortened from about one hour to about 20 minutes, with manual operations reduced to two target inputs.

Furthermore, Agents can combine the traces and metrics provided by performance tools to help determine whether the bottleneck comes from computation, memory access, synchronization, or scheduling, and then choose optimization actions. When a Kernel needs to be modified, it collaborates with FlyDSL to generate candidate implementations and complete compilation, correctness checks, and performance testing on the target hardware.

Iluvatar CoreX Vice President Shi Jiasheng emphasized: "Compilers, runtimes, algorithm libraries, and performance tools remain the technical foundation."

Agents make these accumulated assets easier to invoke, and they also need to rely on them to provide real feedback. For domestic GPUs, changes in software R&D methods have the potential to shorten the cycle of new model adaptation and performance optimization, and to deliver these capabilities to customers faster.

The next-generation product with 600T of edge-side compute power is already on the way

Beyond the cloud, Iluvatar CoreX also showcased its Tongyang edge-side products this time, and used NVIDIA Thor T5000 as the on-site benchmark for comparison.

The test results demonstrated by Iluvatar CoreX show that in large language models such as Qwen3.8-27B and Qwen3.6-35B-A3B, the performance of Tongyang TY1200 reaches 1.2 times that of Thor T5000. In the test of NVIDIA's open-source world foundation model Cosmos-R2-8B, its performance reaches 1.8 times that of Thor T5000.

Tongyang TY1200 provides 300 TOPS of INT8 dense compute power. Similar to cloud products, edge-side competition is also shifting from nominal compute power to the actual runtime performance of specific models and whether devices can meet on-site business needs.

On site, TY1200 demonstrated applications such as offline video generation, contract review, and physical simulation. Among them, at a resolution of 1056×608, generating a 5-second video took about 6 minutes, all completed locally. The demonstration centered on the MicroDuck robotic duck showcased the ability to perform physical simulation, policy training, and action validation on a local terminal.

These applications correspond to different deployment requirements: enterprise contracts and scientific research materials need to stay local, robots need to complete perception and decision-making on site, and some model tasks are also beginning to meet the conditions for edge-side operation.

Song Yuanying, head of marketing for Iluvatar CoreX's Tongyang products, said: "For cloud GPU vendors to do edge-side products, it is not about making chips smaller, but about pushing capabilities downward." The model compression, quantization, and hardware-software co-design capabilities accumulated in the cloud, as well as compilers, operator libraries, and performance tools, have become the foundation for its expansion into the edge-side market.

Currently, the Tongyang series comes in two forms—compute modules and compute terminals—targeting self-integration and finished-product deployment needs, respectively. Iluvatar CoreX also previewed its next-generation edge-side product, Tongyang TY2000, which will reach 600T of dense compute power, further expanding the performance space of edge-side products.

From the cloud-side Tiangai 300, to the Iluvatar CoreX software stack, and then to the edge-side Tongyang series, Iluvatar CoreX has formed a product layout from cloud to edge. The cloud handles large-scale training and inference, the edge stays close to data and business sites, and software extends model adaptation and optimization capabilities to different product forms.

The comparison between Tiangai 300 and H200 demonstrates the competitiveness of domestic GPUs in some high-end workloads. The comparison between Tongyang and Thor extends this competition to the edge side.

A more complete product layout opens more application entry points for Iluvatar CoreX. Whether it can continuously translate performance, software, and delivery capabilities into actual customer output will determine how far this cloud-to-edge layout can go. Leiphone