vLLM 原地重载 LoRA 后新请求仍跑旧权重
(5/7) Stale adapter: after a LoRA adapter is reloaded in place, new requests can still run on the old weights.
Hao AI Lab 用 vstack 智能体形式化验证发现,vLLM 原地重载 LoRA 适配器后,新请求仍可能加载旧权重,而 vLLM 却报告新版本。原因是重载保留了适配器名称与 ID 但指向新文件,重载前已接纳的请求仍指向旧文件,当 vLLM 内存中适配器数量超限需重新加载时会从该旧请求取文件。修复方案是跟踪每个适配器最新一次成功重载,确保从新文件加载。
正文
(5/7) Stale adapter: after a LoRA adapter is reloaded in place, new requests can still run on the old weights.
This breaks adapter updates: once a reload reports success, new requests for that adapter must use the new weights.
An in-place reload keeps the adapter's name and ID but points them to new files. A request admitted before the reload still points to the old files. So when vLLM serves more adapters than it keeps in memory and must load the adapter again, it can load it from that old request, so new requests run on the old weights while vLLM reports the new version.
An in-place LoRA reload while serving left new requests on old weights. Tracking each adapter's newest successful reload ensures vLLM loads from it, so old requests no longer bring back old weights.
PR: https://t.co/Cq8ktHgcGk