AI News

SambaNova and Intel Demonstrate Air-Cooled AI Inference System, SN50 Paired with D-Series GPUs

Unite.AIUpdated Nov 16 14:26

AI summary

SambaNova and Intel demonstrated an air-cooled AI inference system at SuperComputing 2026, using four Intel GPU D-Series accelerators to handle prefill, with the SambaRack SN50 responsible for token generation, running GLM-5.3. The system routes requests through NVIDIA Dynamo, serves the two paths with vLLM, and uses NIXL to transfer KV-cache over standard RoCEv2 Ethernet. SambaNova claims the SN50 can retain KV prefixes in DDR memory to shorten time to first token, but it did not disclose measured latency or throughput data for this combined demonstration.

Why read this

AI score 52

The original article breaks down the hardware division of labor between prefill and decode disaggregation and the standard Ethernet handoff method, offering a reference for data centers constrained by air cooling and power.

Topics

Tags

Body

SambaNova Systems announced on November 15, 2026 that it and Intel had demonstrated an air-cooled AI inference system at SuperComputing 2026, running GLM-5.3 across four Intel GPU D-Series accelerators for prompt processing and a SambaRack SN50 for token generation.

The system divides inference into prefill, which processes an incoming prompt and builds its key-value (KV) cache, and decode, which generates output tokens. Intel’s hardware handles compute-heavy prefill, while SambaNova’s fifth-generation Reconfigurable Dataflow Units (RDUs) handle memory-bound decode. SambaNova presents the recorded system as a design customers will be able to deploy in existing brownfield data centers.

The demonstration is much smaller than Intel’s reference configuration. SambaNova describes an 80-GPU D-Series reference rack delivering 208 petaflops of peak FP4 compute, with 12 TB to 38 TB of memory depending on configuration. Those figures specify the larger prefill design; they are not performance measurements from the SuperComputing setup.

Intel’s D270 product specifications list a 350 W air-cooled GPU with 32 Xe3P cores, 256 XMX matrix engines, standard PCIe 5.0 and up to 480 GB of LPDDR5X memory per card. The standard form factor is intended to work in existing servers without proprietary networking or bespoke systems.

Each SambaRack SN50 contains 16 RDUs and typically requires 20 kW for inference. As many as 16 racks can operate as one unit with up to 256 RDUs, spreading a workload across lower-power racks. SambaNova says its dataflow architecture chains model operations across chips instead of returning data to memory between separate GPU kernels, reducing memory traffic and kernel-relaunch overhead during token generation.

NVIDIA Dynamo sits in front of the two accelerator pools and is designed to route each request toward the hardware already holding its cached context. vLLM serves both the Intel prefill path and SN50 decode path, while NIXL transfers KV-cache data over UCX/RDMA using standard RoCEv2 Ethernet. The handoff can therefore use an established Ethernet fabric rather than a proprietary interconnect between the hardware types.

SN50 retains KV prefixes from earlier prompts in attached DDR memory, while on-chip SRAM and HBM support active decoding. When an agent returns to a conversation or shared context, cache-aware routing can send the request to the SN50 holding that prefix, so only new material goes through GPU prefill and less KV state crosses the network. SambaNova says these cache hits shorten time to first token and free GPU capacity for new requests. The post describes those benefits without reporting measured latency or throughput results for the combined demonstration.

The split also lets providers expand each phase independently. Longer prompts can call for more D-Series prefill capacity, while agents producing longer outputs can require additional SN50 decode capacity, without proportionally expanding the other pool.

That flexibility is aimed at operators whose facilities cannot easily accommodate liquid-cooled systems. Intel says the D270’s standard PCIe form factor and 350 W air-cooled profile are designed for existing racks without a specialized liquid-cooling retrofit. Combined with SN50’s rack-level power profile, the design is intended to add inference capacity within established air-cooled environments.

The demonstration’s significance is the integration: two accelerator architectures, connected by commonly used serving software and standard Ethernet, can be assigned to different inference phases and scaled according to workload. For providers facing fixed power and cooling limits, the remaining question is whether production workloads realize enough of the described caching and utilization benefits to justify operating both pools.

Source

Unite.AI