UniServe 重构 Ulysses：投影权重降至 3.6 GiB
(6/13) Ulysses, reworked: UniServe shards the Q/K/V/gate weights by attention head, all-gathers the narrower hidden states in chunks and projects each chunk as
AI 导读
(6/13) Ulysses 重构：UniServe 按注意力头切分 Q/K/V/gate 权重，分块 all-gather 更窄的隐藏状态，并在每个分块到达时进行投影。 输入集合通信的数据量减少 25%，投影权重从每 GPU 14.4 GiB 降至 3.6 GiB。https://t.co/CC2AfwMdz8
正文
当前显示原文，中文全文译文尚未就绪。
(6/13) Ulysses, reworked: UniServe shards the Q/K/V/gate weights by attention head, all-gathers the narrower hidden states in chunks and projects each chunk as it lands.
The input collective moves 25% less data, and projection weights drop from 14.4 to 3.6 GiB per GPU. https://t.co/CC2AfwMdz8