The More Agents, the Better Intel Looks: Xeon 6+ and KV Cache Tiering Complete the AI Infrastructure

The more agents there are, the more valuable Intel becomes. Original source: Semiconductor Industry Observer. Published: 2026-11-03 16:35. Author: Fang Yuan. Intel chip design generated poster. From sandbox to tiered KV Cache, Intel fills in the gaps in AI infrastructure.

If an enterprise plans to deploy 1,000 agents, what else does it need to prepare besides buying GPUs?

The answer may be more complicated than imagined. Each agent needs an execution environment, memory to save its running state, storage to record context, and a CPU to coordinate tasks and invoke tools. As the number of agents increases, resource consumption that was originally inconspicuous may also become a major expense for data centers. This is also a change taking place in the AI industry. Over the past two years, the industry's competitive focus has mainly been on large models and GPU computing power. But as AI begins to move from answering questions to autonomously executing tasks, the pressure on infrastructure has expanded from pure model computation to execution environments, memory management, data storage and transmission, and other areas.

On October 29, at the Intel session of the 2026 Tencent Global Digital Ecosystem Summit, Intel and Tencent Cloud showcased multiple collaborations centered on Xeon processors, agent sandboxes, memory compression, and KV Cache storage. Notably, on the same day Intel released these technologies, it also announced its financial results for the third quarter of 2026. The company's revenue reached $16.5 billion, up 21% year over year; among that, revenue from the Data Center and AI (DCAI) business reached $7.3 billion, up 78% year over year, making it the business segment with the most obvious growth.

01 Xeon 6+ lands on Tencent Cloud, sandbox concurrency becomes an important breakthrough

At this conference, Intel Xeon 6+ processors were deployed for the first time in Tencent Cloud's new-generation cloud instances.

Xeon 6+ belongs to Intel's Clearwater Forest product line, is based on the 18A process, and has up to 288 energy-efficient cores (E-cores), placing greater emphasis on high-density computing and energy efficiency. Tencent Cloud launched a new generation of CVM cloud instances based on this processor. According to data disclosed on site, overall performance is more than 30% higher than the previous generation, mainly targeting scenarios such as cloud-native workloads, high concurrency, and agent task orchestration.

In addition to processor performance, the more noteworthy collaboration between Intel and Tencent Cloud this time is CubeSandbox. Agents differ from traditional chatbots in that they not only need to generate content, but also need to execute code, invoke tools, and access files. These operations carry potential security risks, so they need to be isolated through independent execution environments. Tencent Cloud's open-source CubeSandbox is an elastic agent sandbox computing platform designed precisely for this need. It uses MicroVMs and combines snapshot and resource reuse technologies, enabling multiple sandboxes to share base images, unmodified memory pages, and file caches, with additional resources occupied only for the parts that change.

For example, if 3,000 virtual machines are created based on the same image, and each virtual machine modifies only 2MB of memory, then with sufficient sharing of base resources, the additional memory usage needs to be only about 6GB. This approach can reduce resource redundancy in traditional execution environments and increase deployment density per machine. On this basis, the two parties also jointly optimized Cgroup, the NIC resource pool, and the OpenCloudOS kernel execution path. According to test results announced on site, while meeting the 200-millisecond SLA requirement, a single machine running CubeSandbox with Xeon 6+ can support 350 concurrent sandbox creation requests, an improvement of about 46% over the comparison platform. At the same time, even under high-load conditions, the P99 latency of sandbox creation remains relatively stable, and as concurrency density increases, latency does not deteriorate significantly. So the value is not just increasing the number of CPU cores, but improving execution efficiency in actual business through joint tuning of processors, operating systems, and sandbox software. For cloud service providers that need to deploy agents at scale, whether execution environments can be created quickly and run stably is directly related to server utilization and deployment costs.

02 IAA and QAT each perform their own roles, memory and KV Cache become new bottlenecks

If the sandbox solves where agents work, then memory and storage solve where the data generated during their work is placed. The longer an agent runs, the more context and task state it retains. Especially when multiple agents execute tasks simultaneously, memory capacity can easily become a limiting factor for server deployment density. To address this issue, Intel and Tencent Cloud demonstrated a memory compression solution based on IAA.

IAA stands for Intel In-Memory Analytics Accelerator, a dedicated hardware acceleration unit integrated inside Xeon processors that can handle specific compression and decompression tasks, reducing the computational burden on general-purpose CPUs. Tencent this time adopted the operating system's Zswap mechanism to compress inactive memory pages, with 2MB huge pages as the test target. Previously, when using software solutions such as LZ4 and ZSTD, multi-channel concurrent decompression was prone to memory bandwidth contention, causing increased latency. Ultimately, the two sides chose the IAA-batch-8 configuration and made lightweight adaptations to the driver and kernel paths. According to the published data, compared with the software compression solution, IAA improves average latency by 16x and maximum latency by 9x. In a comprehensive stress test with 256 concurrent channels, the IAA solution achieved an average latency of 56 microseconds and a maximum latency of 564 microseconds. This shows that it can improve memory utilization while minimizing the impact of decompression on business response. For cloud platforms that need to run a large number of agents simultaneously, memory compression is not only a matter of saving space, but also a matter of whether business latency can be maintained.

Another direction worth noting is KV Cache. KV Cache is an important mechanism for saving context computation results during large model inference. In traditional question-answering scenarios, once the model finishes answering the question, one interaction is basically over. But an agent may continue to execute multiple steps, repeatedly searching, reading files and calling tools, so the context keeps accumulating and the KV Cache also grows larger and larger.

If all caches are placed in GPU memory, both capacity and cost will come under pressure. Therefore, tiered management of KV Cache across GPU memory, CPU memory and SSD is becoming an important direction for inference system optimization. The solution Intel demonstrated this time adopts a storage-compute separation architecture, connecting GPU, CPU and back-end storage through RDMA, and combining self-developed KV Shrink tiered KV software with the QAT accelerator built into Xeon processors to achieve tiered storage, transmission and compression of cached data. Here it is necessary to distinguish IAA from QAT. IAA is mainly aimed at memory page compression, while QAT mainly undertakes acceleration tasks such as data compression and decompression; the two are not the same technology.

In the test, the Qwen2.5-32B-Instruct model was used, with an input of 8000 tokens and an output of 128 tokens, and the test was conducted under a 64GB DDR memory limit. When the KV Cache can achieve a 100% hit rate on back-end high-performance storage, compared with the native vLLM solution without cache reuse, TTFT drops by 90%; at concurrency levels of 8, 16 and 32, throughput increases by more than 5x, 6x and 8x respectively. In addition, after enabling QAT compression, overall throughput does not change significantly, but available capacity increases by nearly 20%.

03 GPU, software ecosystem and on-device intelligence: what is Intel's next card?

In addition to Xeon processors and built-in accelerators, Intel also introduced its GPU product lineup for agent inference. Particularly noteworthy is the GPU D series, codenamed Crescent Island. This product is based on the Xe3P architecture, supports up to 480GB of LPDDR5X memory, has a memory bandwidth of 1.5TB/s, delivers up to 1.3 PFLOPS of FP8 compute, and has a power consumption of about 350W. Unlike traditional GPUs that emphasize peak compute, Crescent Island places more emphasis on memory capacity and energy efficiency. Large-capacity memory can hold more model parameters and KV Cache, making it suitable for long-context, multi-user concurrency and continuously running inference workloads. Intel's published product positioning is that a single card supports running models with up to 300 billion parameters, and an eight-card system supports running models with more than 1 trillion parameters. At present, the product has entered the customer sampling stage.

This reflects another approach by Intel in the GPU market: in the highly competitive compute market, seeking a differentiated market through large memory, relatively low power consumption and the air-cooled deployment conditions of existing data centers. However, competition in the GPU market depends not only on hardware specifications; the software ecosystem is equally important. Intel is reducing the cost of migrating CUDA projects to the Intel platform through open software technologies such as SYCL and SYCL-TLA, as well as AI Agent-assisted code migration tools. In the experiment demonstrated on site, 10 different projects completed preliminary migration through large models and dedicated Skills, after which further operator optimization is still needed.

In the terminal market, Intel is also advancing hybrid on-device and cloud intelligence. This conference showcased the WorkBuddy and Marvis local agent solutions developed in cooperation with Tencent. Marvis provides nearly 20 local skills, while WorkBuddy integrates more than 30 local or hybrid experts, and some tasks can be completed through local models and OpenVINO acceleration. Therefore, Intel's AI layout is not limited to data centers; it also hopes to extend model inference, task execution and privacy protection capabilities to AI PCs. The cloud handles complex reasoning, while the terminal handles some tasks with low latency and high privacy requirements. This hybrid model also brings new application space for PC processors, integrated GPUs and on-device AI software.

04 Nvidia and AMD are also stepping up

Intel is not the only vendor that has seen this market change. Nvidia and AMD have also recently readjusted their product layouts around agents, and the direction of both companies has extended from single GPU products to CPUs, storage and system architecture.

In addition to Intel, Nvidia also dedicated a share at this conference to AI infrastructure. Nvidia introduced five major network infrastructures for AI factories: Scale-Up, Scale-Out, Scale-Across, Context Memory and Scale-In, corresponding respectively to high-speed GPU interconnect, server cluster expansion, cross-data center connectivity, context storage and infrastructure management. Among them, Context Memory is particularly noteworthy. As agent tasks continuously accumulate context, the scale of KV Cache keeps growing, and Nvidia hopes to use network and storage architecture to allow different compute nodes to share and access cached data, reducing redundant computation. This coincides with Intel's direction of optimizing KV Cache through QAT and KV Shrink, except that Nvidia places more emphasis on network interconnect and cluster-level resource management. The introduction of technologies such as BlueField-4 and DOCA also shows that DPUs are moving beyond traditional data transmission and offload tools to further participate in data access, security and management of AI infrastructure.

AMD brought another noteworthy direction: on-device AI and storage collaboration. Together with partners such as Longsys and Liulian Intelligence, it showcased an agent host solution based on Ryzen AI Max+ 395, combined with Longsys's AIDIMM high-bandwidth memory, iSA storage agent and AISSD, to solve memory capacity and data scheduling problems when running large models locally. Among them, AIDIMM uses LPDDR5X dies and a 256-bit width design, while iSA coordinates data flow between DRAM and SSD through methods such as MoE expert dynamic offloading, cache tiering and data prefetching. Joint tests published by Longsys show that a 64GB AIDIMM configuration can support local operation of 70B, 80B and 122B-level models, and the related solutions also extend to the AMD Ryzen AI Halo developer platform equipped with AISSD and iSA. The significance of this design is that it no longer requires all model data to reside in DRAM, but instead allocates data to different storage tiers according to runtime requirements. For on-device devices with limited memory capacity, this provides a new approach to running larger-scale models.

From KV Cache management in the cloud to memory-SSD collaboration on the device side, it can be seen that storage is transforming from traditional supporting hardware into an important component affecting AI inference efficiency. Nvidia focuses on high-speed connectivity and cache sharing within data centers, while AMD and its ecosystem partners are exploring how to break through the memory limits of on-device devices through storage tiering. The two different paths actually point to the same trend: competition in AI infrastructure is expanding from compute chips themselves to areas such as memory, storage and data flow.

05 Conclusion

From the showcase at this Tencent Global Digital Ecosystem Summit to Intel's latest third-quarter financial report, a noteworthy change can be seen: data centers are once again becoming an important growth source for Intel, and agents may further expand demand for infrastructure such as server CPUs, memory and storage.

For the entire semiconductor industry, the changes brought by agents may be more profound than a simple round of GPU upgrades. When AI begins to continuously execute tasks, what determines infrastructure costs is no longer just how many tokens the model can generate per second, but also the creation speed of the execution environment, the storage cost of context, the data transmission efficiency, and the energy utilization of the entire system.

In the past, the semiconductor industry competed over who could provide stronger computing power; in the agent era, competition is shifting to who can convert computing power into more real work output at a lower system cost.

This content is original to the platform, and the copyright belongs to the platform. Reproduction is not permitted without permission. To reprint, please contact the platform. Comments: No user comments yet. Latest News More Related Recommendations

Global smartphone shipments plunge 19%, revenue rises 3% against the trend 2026-11-03 16:28

Apple's robot is not on a humanoid path 2026-11-03 16:24

onsemi acquires Ferric for $600 million, AI chip power supply battle spreads inside packaging 2026-11-03 16:17