AI News

Mapping Four 32 GB Resizable BAR GPUs Through a PCIe Switch on Linux

Hacker News TrendingUpdated Nov 13 00:24

AI summary

The author connected four AMD Radeon AI PRO R9700 cards and two RTX 3090 cards behind a Broadcom PEX88096 PCIe Switch, and used a self-developed kernel module, gpu-rebar-layout, to set each R9700's BAR to 32 GB and rescan before the GPU driver loaded, so that all four cards received a full 32 GB BAR, with amdgpu reporting BAR=32768M. In Linux 7.0, pbus_size_mem() sums by child device size and thus misses the 32 GB alignment gaps, resulting in "can't assign; no space"; Ilpo Järvinen's 14-patch fix series has reached v5 and has not yet been merged. With P2P enabled, R9700 cards communicate pairwise at 14.0 GB/s, and simultaneous ring copying across four cards reaches 54.7 GB/s.

Body

Only a summary is authorized here. Continue at the validated source link.

Source

Hacker News Trending