AI Trends

Tencent Hunyuan Team Discusses Hy4 Compute, Architecture, Evaluation, and Open-Source Roadmap

Phoenix TechnologyUpdated Nov 21 09:17

AI summary

In a Reddit AMA, the Tencent Hunyuan team said it would use its existing computing resources to catch up with the frontier, while the next generation of Hy models will be scaled up and continue to receive investment in large-scale reinforcement learning. The team also explained why Hy4 uses MLA and confirmed that Hy4-mini, Flash-Lite, and a model designed for devices with 24GB of VRAM are being planned or developed. The team said it would continue to release open weights and addressed questions concerning long chains of thought, benchmark evaluations, tool use, and limitations in cybersecurity capabilities.

Why read this

SelectedAI score 76

This AMA highlights Tencent Hunyuan's technical assessments of compute constraints, model scaling, architectural trade-offs, evaluation methods, and open-source security.

Topics

Tags

Body · AI translation

Tencent’s Yao Shunyu Team Responds to Everything Late at Night: Insufficient Computing Power, Models Gaming the Rankings, and Lei Ting’s Big Thoughts — Download the Zhidx App for an Exclusive First Look — November 21, 2026, 08:30:07, from Beijing

Zhidx reported on November 21 that in the early hours of this morning, Tencent’s Hunyuan team held an AMA on the social media platform Reddit, engaging with developers around the world about the Hy4 model series. Yao Shunyu also appeared in the comments, though we do not know which alias he was using.

▲Yao Shunyu previews the Reddit AMA on X (Image source: X)

This AMA took place at a very interesting moment.

Just yesterday, Tencent released the weights of its flagship Hy4 model and published its model safety evaluation card (System Card). A few days earlier, Tencent had also launched Hy4 flash, which focuses on low cost and high efficiency. The successive releases of these two models have made Tencent Hunyuan’s technical approach a focus of attention in the community.

Meanwhile, competition in the AI industry is also changing. Recently, third-party model evaluation organization Artificial Analysis substantially revised its evaluation system, causing fluctuations in the scores of some Chinese models. Against the backdrop of restricted access to high-end computing power and the continued evolution of leading global models, developers are also concerned about how Chinese models can narrow the gap with the best models.

Zhidx also participated in the AMA and asked Tencent’s Hunyuan team: As access to advanced AI chips becomes increasingly restricted, will Tencent’s Hy models be able to continue narrowing the gap with leading global models in the future?

A member of Tencent’s Hunyuan team responded: “We are making the fullest possible use of our existing resources to catch up with the frontier.”

▲The Hunyuan team answers Zhidx’s question (Image source: Zhidx)

In addition to the computing-power issue, Tencent’s Hunyuan team disclosed a considerable amount of new information during the discussion.

Regarding model size, the Hunyuan team stated clearly that the next generation of Hy models will further increase the parameter count while continuing to invest in large-scale reinforcement learning.

For smaller models, the team revealed that products including Flash-Lite and a model for devices with 24GB of VRAM are being planned.

On the technical front, Tencent also disclosed details about some of Hy4’s architectural choices for the first time during the discussion. The team repeatedly weighed the GQA and MLA attention mechanisms before ultimately choosing the MLA architecture because DSA reduced MLA’s computational overhead during decoding.

Tencent’s Hy team also responded to questions concerning model evaluation, safety restrictions, the AI coding ecosystem, and whether models can achieve recursive self-improvement (RSI).

Hy4 is Tencent’s latest-generation flagship model, with particular improvements in capabilities such as long-horizon software engineering, tool use, visual understanding, and scientific research. Hy4 flash further optimizes inference costs. It has 224B total parameters, including 64B Engram parameters, and activates only 12B parameters per token.

According to data disclosed by Tencent, in its internally developed WorkIndex benchmark for everyday work tasks, Hy4 flash’s cost per task is only 1/20 that of Hy4. In a blind test involving 85 internal users and 259 work tasks, its cost per task was approximately RMB 0.26.

However, Hy4 still trails the world’s leading closed-source models on some highly challenging tasks. The Hunyuan team must still address how to continue improving model capabilities with limited computing power while balancing inference costs, product experience, and the open-source ecosystem.

The following are the key points Zhidx compiled from the AMA. For ease of reading, we have edited them without changing their original meaning:

Full Q&A:

I. Catching Up with the Frontier Using Limited Computing Power: The Next-Generation Model Will Be Significantly Scaled Up, and Smaller Models Are on the Way

(1) Zhidx asks: As access to advanced AI chips becomes increasingly restricted, what is currently the biggest bottleneck facing Tencent’s Hy models? Is it computing power, training efficiency, or algorithms? Will the Hy series be able to continue narrowing the gap with the world’s leading models in the future?

Tencent Hunyuan: We are making the fullest possible use of our existing resources to catch up with the frontier.

(2) A user asks: Are there plans to increase the parameter count of Hy4.5 to between 1.5 trillion and 2 trillion in the future?

Tencent Hunyuan: The next generation of Hy models will definitely be scaled up significantly.

(3) A user asks: Do you believe the current Hy series is already approaching its capability ceiling? Will stronger models in the future necessarily require more parameters, or could you maintain the current scale and further improve capabilities through post-training?

Tencent Hunyuan: We believe Hy4 still has substantial room for improvement, particularly in post-training. We are investing heavily in large-scale reinforcement learning (RL) while continuously improving the user experience and model safety. Please stay tuned for further updates. We also plan to train larger-scale models. We firmly believe in the importance of scaling and will continue pursuing this direction.

(4) A user asks: Companies such as ByteDance and Kimi appear to be moving toward ultra-large-scale models with 3T or even 5T parameters, while Tencent’s Hy series remains at around 700B. Why did you choose this path? How do you view the impact of parameter count on a model’s level of intelligence? Do you plan to scale up the models further in the future?

Tencent Hunyuan: Hy3 and Hy4 have already demonstrated that smaller but well-trained models can often outperform larger ones. More importantly, smaller models cost less, while also allowing us to learn more and accumulate more experience during R&D.

(5) Questions from multiple users: Will Tencent launch Hy4-mini and dense models in the 24B–31B range in the future? Are there plans to launch smaller models capable of running on devices with 24GB of VRAM or 64GB–128GB of system memory? What plans are there for models with fewer than 100B parameters that are specifically optimized for programming tasks?

Tencent Hunyuan: Hy4-mini is already on the development roadmap, and more small models will be launched in the future. The team is considering dense models in the 24B–31B range. Even smaller dense models may also be launched, but there are currently no plans to specifically develop models with fewer than 100B parameters that are optimized for programming tasks.

Regarding local deployment, the Hunyuan team confirmed that it will launch models for devices with 24GB of VRAM and will continue developing models suitable for devices with 64GB–128GB of system memory. The team also confirmed that it plans to release open-weight models similar in size to the current models that, after Q4 quantization, will be suitable for devices with 128GB of memory.

In addition, Hy4 flash will continue to receive updates, and the team is optimizing pre-training and post-training approaches for Flash-class models. As for the positioning of Hy4 and Hy4 flash, the team said both are general-purpose models and were not trained specifically for programming tasks.

(6) Questions from multiple users: Beyond language models, will Tencent launch smaller open-source image models and upgrade its video generation and translation models? Can Hy4 support dialect recognition and real-time speech translation? What progress has been made in continual learning and artificial general intelligence (AGI)?

Tencent Hunyuan: For image generation, the team said it is considering launching smaller image models. It also suggested that developers keep an eye out for news about its video models in the near future, but did not disclose a specific release date.

Regarding translation, the team confirmed that the Hy-MT model will soon be upgraded and said it would seriously consider users’ suggestions concerning the translation of Chinese online novels.

As for dialect recognition and real-time speech translation, the Hunyuan team acknowledged that Hy4 does not currently support audio understanding or audio generation. However, users can try the automatic speech recognition (ASR) and text-to-speech (TTS) features offered by Tencent Yuanbao, which may be able to meet these needs.

Regarding continual learning, the team believes this is a highly promising research direction and is actively conducting related research, but it did not disclose any specific results.

In addition, one user cited an AGI capability classification framework and asked what level Hy4 has currently reached. A Hunyuan team member offered a personal assessment: “Around Level 2.5.”

▲The AGI capability classification framework proposed by Google DeepMind (Image source: Reddit)

II. Architectural choices involved repeated trade-offs; greater MoE sparsity is not always better

(7) A user asks: Does Tencent plan to launch a small Flash-Lite model for low-latency tasks? What advantages does iHC have over mHC and gated residuals (GR)? Why did Hy4 flash choose sliding-window attention (SWA), and will you experiment with other attention mechanisms such as QSA in the future?

Also, will the licenses for older open-weight models on Hugging Face be updated to support users in the European Economic Area (EEA)?

Tencent Hunyuan: Great questions—there are a lot of technical ones here.

First, yes. Please stay tuned for a technologically advanced Flash-Lite model.

Second, compared with mHC and GR, iHC is much simpler and can achieve similar performance at a lower cost.

Third, SWA can reduce the KV Cache size by a factor of four, significantly lowering inference serving costs.

Fourth, of course! We are open to all kinds of architectural innovations.

(8) A user asks: What is it like to participate in the R&D of open-weight models and contribute to the open-source community?

Tencent Hunyuan: Contributing to the open-source community is incredibly rewarding, especially when we see so many people taking an interest in our work.

(9) A user asks: What was the most difficult moment you personally experienced during the R&D process?

Tencent Hunyuan: For me, the hardest part was scaling from Hy3 to Hy4.

We did not conduct as many detailed ablation studies on the training data as I would have liked, so there was still a great deal of uncertainty when training began. To be honest, I was extremely nervous at the start of training.

Fortunately, the training process ultimately went unexpectedly smoothly. It felt like making an all-or-nothing bet and actually winning.

(9) A user asks: If you could make one wish, what would you want future large language models to do for you?

Tencent Hunyuan: As a true food lover, I hope future large language models will be able to create entirely new recipes and genuinely ensure that the finished dishes taste exactly the way I like them.

(10) A user asks: Aside from the final Hy4 architecture, what was the most competitive alternative architecture that Tencent’s Hy team seriously considered but ultimately abandoned? Why was it abandoned?

Tencent Hunyuan: We previously went back and forth between GQA and MLA.

MLA is significantly more compute-intensive than GQA, making compute capacity the performance bottleneck when using speculative decoding on inference GPUs.

However, with the same KV cache size, MLA can deliver better model quality than GQA.

Fortunately, the introduction of DSA allowed us to reduce MLA’s computational intensity during decoding. This ultimately shifted the trade-off between the two architectures and led us to choose MLA.

(11) A user asks: As the original Hy4 architecture evolved into Hy4 flash, how did you decide which network layers or architectural components to retain? What primarily guided each decision to adjust the neural network layers?

Tencent Hunyuan: Hy4 flash inherits several technologies from Hy4, including iHC, gated MLA, and DSA, while introducing new architectural components such as hybrid SWA, LatentMoE, and Engram.

These design choices were supported by the extensive results the team accumulated through scaling experiments at different model sizes, as well as the co-design of the model architecture and inference system.

(12) A user asks: How important is additional parameter density to a model’s intelligence? Have you tested configuring Hy4 flash with 156B total parameters and 8B or 5B active parameters? How much would model quality decline?

Tencent Hunyuan: We conducted scaling experiments with different levels of MoE sparsity, similar to the research in the MAI-Thinking-1 paper, and reached roughly the same conclusions.

With the number of active parameters held constant, increasing sparsity can improve computational efficiency. Therefore, this is a viable technical path.

However, 156B-A8B or A5B represents a completely different trade-off. This approach reduces the number of active parameters while keeping the total parameter count unchanged. I expect that it would significantly impair model quality.

The number of active parameters essentially determines the model’s compute budget. Increasing sparsity can recover some computational efficiency, but it cannot fully offset the impact of reducing the active parameter count.

Ultimately, fewer active parameters mean less compute and a less capable model.

(13) A user asks: I have always hoped that a lab would push MoE sparsity to new heights—for example, by developing a model with 200B total parameters and 3B active parameters, perhaps combined with Engram. It might perform very poorly or unexpectedly well, but inference should be fast. Just how sparse can an MoE architecture get?

Tencent Hunyuan: In my view, an excessively sparse MoE architecture is not conducive to inference. Such an architecture consumes a great deal of GPU memory without fully utilizing the GPU’s computational capacity (FLOPs).

In fact, we could build a model with the same total parameter count but more active parameters while achieving the same inference throughput.

(14) A user follows up: That is certainly true for high-concurrency inference services in data centers. But the situation is different for local deployment. Limited by memory bandwidth, A12B models run slowly on Strix Halo and DGX Spark devices, while some models with fewer active parameters perform quite well.

Tencent Hunyuan: I understand your point. For on-device models, we will take a different design approach, co-designing the model architecture more closely around the characteristics of edge-device GPU hardware.

(15) A user asks: How did you determine the allocation ratio between the approximately 63.5B Engram parameters and the approximately 155.5B neural network backbone parameters in Hy4 flash? The Engram paper mentions that a ratio of around 20%–25% was ideal for its models. Did your experiments reveal a similar pattern?

Tencent Hunyuan: We found that the optimal balance—the lowest point of the U-shaped curve—is achieved when Engram parameters account for approximately one-third of the total parameter count, including Engram itself.

(16) A user asks: If Hy4 were allowed to operate completely autonomously, is it currently capable of developing the next generation of models through recursive self-improvement (RSI)? Have you established relevant benchmarks that go beyond simple runtime and tool-call counts to evaluate a model’s genuine autonomous R&D capabilities?

Tencent Hunyuan: Our RSI capabilities are still at an early stage. We have developed several internal evaluation benchmarks to assess key RSI capabilities. In addition, we are developing Hyra 3.0 and continually creating better reusable tools for model development.

(17) A user asks: What will be the primary path for improving the capabilities of future Hy-series models?

Tencent Hunyuan: Our current top priority is to build a solid foundation, improve model intelligence, and work closely with our products to provide a better user experience.

(18) A user asks: Like Hy4 Preview, the official version of Hy4 sometimes also spends too long reasoning. Is this overthinking problem caused by the design of the reinforcement-learning reward mechanism, or is it related to the training-data distribution? Can the model learn to stop thinking at the appropriate time?

Tencent Hunyuan: Hy4’s goal is to maximize model intelligence, so longer chains of thought (CoT) are a known issue.

However, this also allows us to transfer Hy4’s capabilities to Hy4 flash through distillation and achieve good cost efficiency. For Hy4 flash, we use reinforcement learning to ensure that its chain of thought remains relatively short.

(19) A user asks: Command-line tools (CLIs) are prone to errors when models fabricate incorrect command syntax or parameter flags. How do you use reinforcement learning to improve tool-calling accuracy while preventing the model from losing context?

Tencent Hunyuan: To improve tool-calling accuracy, we have invested considerable effort in reinforcement-learning data, including difficult cases in which models are prone to making mistakes.

We also use a fine-grained reward mechanism that imposes harsher penalties for errors in critical details.

For context retention, we are continuously extending the context length used in reinforcement-learning training and specifically improving the model’s context-compression capabilities.

(20) A user asks: Without disclosing internal information, could you describe the main scenarios on which Hy4’s agentic programming tasks focus?

For example, 3D modeling, large monolithic code repositories, short-duration zero-shot programming, documentation and code updates, the comprehensibility of code, comments, and file structures, and code cleanliness during multi-turn development. Can the model autonomously perform hundreds of operations without human intervention while maintaining code quality?

Tencent Hunyuan: We cover all of the scenarios mentioned above and place great importance on task diversity.

(21) A user asks: How does Hy4 flash determine the quantization precision for different network layers and experts? How do you ensure the stability of the MoE router during quantization? If the number of active parameters is reduced further, which model capabilities would be affected?

Tencent Hunyuan: We assign precision based on each network layer’s sensitivity to quantization. For example, down projections are generally more sensitive and therefore require more bits. Rather than setting precision separately for each expert, we prefer tensor-wise allocation to improve compatibility with inference engines.

During quantization-aware training (QAT), we train the MoE router together with the rest of the model to keep the router stable. Compared with approaches such as LoRA, we prefer full-parameter QAT for better model quality, although we have not yet conducted LoRA-related experiments.

If the number of active parameters were reduced from 12B to 3B, we expect reasoning depth, long-context tracking, and tool-calling reliability to be the first capabilities to deteriorate.

In addition, Engram parameters are not included in the 12B active parameters. Each layer of the model uses an independent N-gram table, and where the memory budget permits, we prioritize model capabilities.

(22) A user asks: Xiaomi’s MiMo team recently disclosed its model-training process in detail. What do you think of this approach? If you can share, is MiMo’s post-training method similar to yours, or are there significant differences in training costs, data scale, the proportions of different task types, and other areas?

Tencent Hunyuan: We are glad to see the MiMo team sharing its research. Our reinforcement-learning training is conducted at a larger scale.

(23) A user asks: Is it possible that models using a recurrent MoE architecture will be released in the future?

Tencent Hunyuan: It is entirely possible. Stay tuned.

(24) A user asks: I saw Hy4 flash’s test results on a large number of PHP code repositories. It performed some Shell and DevOps tasks very well, including specialized scenarios such as ZFS dataset migration. How did you address the special challenges of PHP type inference?

Tencent Hunyuan: Better training data, diverse internal evaluations, and “non-benchmaxing.”

(25) A user asks: Why did you not publish scores for common benchmarks such as GPQA Diamond, τ-bench, IFBench, and SWE-bench? Is there a specific reason?

Tencent Hunyuan: We presented results from more recent and advanced benchmarks, such as Agents’ Last Exam and ProgramBench. The benchmarks you mentioned were released relatively early, and their scores are already approaching saturation.

(26) A user asks: Do you plan to train a Ternary Model from scratch to substantially improve inference-serving efficiency?

Tencent Hunyuan: At present, we have no plans to train a ternary model from scratch.

(27) A user asks: How much attention have you devoted to creative writing and content readability? A model that explains issues more clearly and accessibly would be one of the main reasons I might stop using Qwen3.8 Flash Next and switch to another model.

Tencent Hunyuan: We likewise believe that creative writing and readability are extremely important, and we are making them key areas for improvement. We have already done considerable work in this area, but there is still room for improvement. Sometimes, one good example is worth five paragraphs of text.

We care deeply about how the model performs in real-world use, and writing quality and readability are both important parts of that. We are actively improving both capabilities.

(28) User question: I have noticed that large language models’ writing abilities have declined. I used to use Claude 3.5 to edit text, but I no longer use Claude 5.5. I have found that only the code documentation generated by Gemini Pro barely meets my requirements. Qwen3.6 also writes better than Qwen3.8. Is this because large models are increasingly focused on code training, because of overfitting, or for some other reason?

Tencent Hunyuan: We have noticed this phenomenon as well. We believe it may be related to the models’ training objectives.

We want our models to write more like humans and express themselves more naturally, rather than producing formulaic, template-like content. We are working hard to improve this, so please stay tuned.

III. We Will Continue to Embrace Open Source, and No Single Leaderboard Can Determine a Model’s Capabilities

(29) User question: Will Tencent remain committed to open source, or will it shift toward closed-source models like Meta? If Tencent develops AGI or the most capable AI model in the future, will it still choose to make it open source?

Tencent Hunyuan: For now, yes, we will remain committed to open source.

(30) User question: The Artificial Analysis leaderboard is clearly biased toward closed-source US models. To what extent has this harmed the commercial interests of Tencent Hy and other leading Chinese model labs?

Tencent Hunyuan: A single website cannot truly determine whether a model will be adopted in the real world. We consider a variety of evaluations, including internal benchmarks, public evaluations, third-party evaluations, and feedback from real users.

To properly assess a model’s performance, you need to look at the full set of evaluation results. Ultimately, that is what users truly care about.

(31) User question: Besides communicating through research papers, do you also communicate with other open-source large model labs?

Tencent Hunyuan: We are actively learning from the open large model community, and we also hope to give more of our research findings back to the community.

(32) User question: How does your team use large language models in its daily work? Do you use open-source models, Tencent’s own internal models, or other closed-source models?

Tencent Hunyuan: We use excellent models regardless of whether they are open source. Of course, the Hy series is also widely used internally.

(33) User question: Does Tencent plan to release quantized versions for different hardware configurations or participate in projects supporting deployment with limited VRAM, so that Hy4 can also run on devices with limited VRAM? Also, do you feel pressure to restrict or remove models’ cybersecurity capabilities, as Kimi has done?

Tencent Hunyuan: We are working on adapting Hy4 to Strata, and the relevant support will be available soon. To release the model responsibly as open source, we have restricted some of its cybersecurity capabilities. We believe this is necessary.

(34) User question: EXL3 quantization is very helpful for local inference on lower-end hardware. Does Tencent plan to support this quantization format?

Tencent Hunyuan: At present, we have no official plans for an EXL3 version. However, since the model weights have been made available, we believe the open-source community will soon begin releasing corresponding versions.

(35) User question: I have only just started learning about this model and have not tried it yet. What is its parameter count, and what are its RAM and VRAM requirements?

Tencent Hunyuan: You can run the Hy4 flash model on an M5 Max and a Spark device with 128GB of memory.

(36) User question: If the ultimate goal is to build on-device Agents, would you recommend that developers start now by running smaller local models on a PC or Mac, building an Agent Harness, and testing real-world use cases?

Tencent Hunyuan: We recommend starting now. I believe local deployment will always face efficiency challenges, while the relevant technologies will continue to evolve.

(37) User question: Will Tencent prioritize investment in reinforcement learning for scientific research, particularly biology tasks, in the future? Especially for models between Hy4 flash and the 500B-parameter scale.

Tencent Hunyuan: We are actively investing in reinforcement learning for scientific research, while also placing a high priority on safety.

(38) User question: What do you think will usher in the next era of artificial intelligence? Will it be a breakthrough in model architecture, more or higher-quality training data, or something else?

Tencent Hunyuan: We have always believed that data and evaluation are the most important tools for achieving artificial general intelligence (AGI). Data determines which capabilities a model can acquire, while evaluation guides us in the right direction.

(39) User question: How should someone enter the field and become an AI engineer today?

(40) User question: Setting aside model benchmark scores and judging instead by factors such as the teams’ technical expertise and research quality, how would you evaluate other open-source model labs such as DeepSeek, Zhipu AI, Moonshot AI, and Alibaba? If you had to choose a top three, how would you rank them?

Tencent Hunyuan: I’m not falling for that one. They are all excellent.

IV. Proactively Restricting Some Cybersecurity Capabilities; Hy4 Is Relatively Strong at Vulnerability Discovery but Relatively Weak at Exploitation

(41) User question: Under normal circumstances, extensive safety alignment training causes open-source models to refuse basic penetration testing, reverse engineering, or CTF tasks. How did you prevent the model from over-refusing these dual-use programming requests?

Tencent Hunyuan: In cybersecurity training, we focus on defensive tasks and vulnerability analysis and remediation, while excluding data involving vulnerability exploitation and offensive attacks. We also conduct red-team evaluations to assess potential risks.

Avoiding excessive refusals remains an important area for improvement.

(42) User question: Hy4’s system card mentions that Tencent deliberately filtered its cybersecurity training data, retaining only data related to vulnerability reproduction and remediation while removing offensive training content such as vulnerability exploitation and reverse engineering, thereby limiting the model’s cybersecurity capabilities. Does this instead come at a cost to defenders?

Tencent Hunyuan: According to our evaluations, Hy4 is relatively strong at vulnerability discovery but relatively weak at vulnerability exploitation.

(43) User question: Why remove cybersecurity red-team content from the training data? Doing so may weaken the model’s capabilities without delivering any real safety benefits. Have you considered other alternatives?

Tencent Hunyuan: Filtering this data is a more fundamental measure, which must also be combined with trained safety safeguards that refuse relevant requests when necessary. These measures are all intended to enable responsible open sourcing.

(44) User question: Have you ever used past AI alignment failures—whether encountered by Tencent itself or publicly disclosed by other labs—as model training data or evaluation targets?

Tencent Hunyuan: These cases are used only during evaluation for tracking and monitoring purposes.

(45) User question: When running models internally—for example, during training, evaluation, and deployment, particularly for agent tasks—what network access controls do you implement?

Tencent Hunyuan: We use allowlists and blocklists. During model evaluation, because multiple attack methods targeting frontier models have already been identified, we need to repeatedly fix and adjust the testing environment and the relevant access-control lists.

In addition, throughout the training and evaluation process, we continuously monitor for potential cyberattacks and model alignment issues, and remediate the relevant environments when problems are discovered.

“Special notice: The content above (including any videos, images, or audio) was uploaded and published by users of Dafeng Hao, a self-media platform operated by Phoenix New Media. This platform merely provides information-storage services.

Notice: The content above (including the videos, pictures and audios if any) is uploaded and posted by the user of Dafeng Hao, which is a social media platform and merely provides information storage space services.”

Source

Phoenix Technology