Apple 用 MLX 在 M5 GPU 神经加速器上探索 LLM 推理
Apple 在搭载 M5 芯片的 14 英寸 MacBook Pro 上,通过 MLX 框架利用 M5 GPU 的神经加速器运行 LLM 推理。基准测试显示,M5 相比 M4 在首个 token 生成时间(TTFT)上实现 3.33-4.06 倍加速,后续 token 生成速度提升 19-27%,得益于内存带宽从 120GB/s 提升至 153GB/s。测试涵盖 Qwen3-1.7B/8B/14B/30B-A3B 及 gpt-oss-20b 等模型,24GB 统一内存可轻松承载 8B BF16 或 30B MoE 4-bit 量化模型。

licon platform. Running LLMs on Apple Silicon MLX LM is a package built on top of MLX for generating text and fine-tuning language models. It allows running most LLMs available on Hugging Face. You can install MLX LM with: pip install mlx-lm And, you can initiate a chat with your favorite language model by simply calling mlx lm.chat in the terminal. MLX natively supports quantization, a compression approach which reduces the memory footprint of a language model by using a lower precision for sto
匹配字段正文