MacroSAN Showcases at the 2026 Data Storage Industry Conference: Alpha700 Accelerates Large Model Inference with KV Cache Offloading

From October 28 to 29, the 2026 Data Storage Industry Conference, hosted by the China Electronics Standardization Association and organized by the Data Storage Professional Committee of the China Electronics Standardization Association, was held in Zhengzhou. As an outstanding member unit of the Professional Committee for 2026 and a leading AI storage enterprise, MacroSAN Technology was invited to attend the conference and join representatives from government, industry, academia, research, and application sectors in discussing AI storage technology innovation and industrial collaboration.

Co-building Standards: Special Achievements Unveiled in a Concentrated Showcase

Currently, AI-driven storage has entered an ultra-long cycle, and the core contradiction in the industry has shifted from "whether computing power is sufficient" to "whether data can feed it adequately." The problem of idle computing power in GPU clusters caused by data waiting has become increasingly prominent.

Facing the new stage of development, the conference officially released the Future Storage special research achievement—the "Action Initiative for High-Quality Development of the AI Storage Industry"—which systematically reviews and presents forward-looking research conclusions and development paths in the AI storage field.

English text is being prepared.

As a core co-drafting unit, MacroSAN Technology, based on extensive practical experience, fed frontline practices such as KV Cache offloading, tiered storage capacity, and domestic computing power adaptation back into the standards drafting work, helping the industry unify technical specifications, promoting industrial compatibility and interoperability, and providing standards guidance for the construction of intelligent computing infrastructure across various industries.

Conference Presentation: KV Offloading Breaks Through the VRAM Bottleneck of AI Inference

With the large-scale deployment of large-model Agents and long-context services, KV Cache is rapidly encroaching on GPU HBM VRAM, creating a key bottleneck that constrains inference concurrency and drives up service latency. Expanding HBM alone makes it difficult to balance capacity, cost, and efficiency, so external storage has thus become a key link in unleashing inference performance.

To address this challenge, Shan Donglai, AI Storage Product Director of MacroSAN Technology, delivered a keynote speech titled "Application Practices of External Storage in KV Offloading," sharing MacroSAN Alpha700 AI dedicated storage's innovative practices in accelerating large-model inference.

Its core approach is to enable external storage to move from a "capacity carrier" to an "inference accelerator" through tiered KV offloading. As the product implementing this approach, MacroSAN Alpha700 AI dedicated storage can simultaneously support two modes: L3.5 low-latency offloading and L4 high-throughput shared offloading. The former uses DPU, NVMe-oF over RoCE, and data direct pass-through to achieve low-latency cache reuse, serving multi-Agent and multi-turn calls; the latter builds a shared KV resource pool through a parallel file system, GDS, and RDMA to support high-throughput scenarios such as long-context AI programming.

Practical data shows that in the application of an intelligent agent platform at a large manufacturing enterprise, the L3.5 solution increased the global KV Token hit rate from 38% to 79%, reduced TTFT P50 from 1,200 milliseconds to 520 milliseconds, and lowered Agent response time by more than 40%; the L4 solution increased the hit rate from 22% to 91% and reduced code generation time by more than 45%, sparking heated discussion among the guests at the event.

On the basis of technological path innovation, MacroSAN Technology continues to expand into the domestic intelligent computing ecosystem and has partnered with Hygon Information to create the "Alpha700 + Hygon DCU accelerator card" AI storage solution: Alpha700 relies on its built-in parallel file system and MSCache KV cache scheduling capabilities to build a PB-level shared cache pool, effectively improving cache hit rates and reducing redundant computation; Hygon DCU supports inference computing power loads with large VRAM and high-throughput computing power. Through collaborative optimization, the two can significantly reduce first-token latency and greatly improve inference concurrency and throughput efficiency.

This joint solution is deeply adapted to mainstream large models such as DeepSeek and Tongyi Qianwen, is ready to use out of the box, and can provide an independently controllable and extremely stable storage foundation for scenarios such as intelligent computing platforms, intelligent risk control, and intelligent customer service, supporting the efficient implementation of AI applications across various industries.

From product innovation to co-building standards, MacroSAN Technology continues to deepen its industrial influence. In the future, MacroSAN Technology will continue to work with upstream and downstream partners in the industry chain to accelerate the deep integration of storage and computing and help thousands of industries move toward a new future of digital-intelligent symbiosis.