Google 发布 LiteRT Tensor API,支持用 C++ 手写端侧模型图
Google 发布 LiteRT Tensor API,让开发者在 C++ 中逐算子构建计算图,同一份代码可经 LiteRT 跑在 CPU、GPU(ML Drift)和 NPU 上,或不经模型文件直接通过 XNNPACK 在 CPU 运行。官方用该 API 手写 SAM 2 并部署到浏览器 WebGPU,在 MacBook Pro 上比 Apple MLX 快约 1.5 倍,输出掩码与 Meta PyTorch 参考逐像素一致。该 API 还支持从 .safetensors 直接加载权重,并让预处理、推理与后处理通过共享加速器缓冲区串联,避免数据回传 CPU。

or-optimized LLM composite ops, such as scaled dot-product attention, RMSNorm, and KV-cache updates. Direct weight ingestion: load weights straight from model checkpoints, such as Hugging Face .safetensors files, without conversion scripts needed in between. Tensor-centric buffer interop: pre-processing, model partitions, and post-processing pass tensors to each other through shared accelerator buffers, so intermediate data stays on the accelerator with zero copies. Building blocks for AI agents
匹配字段正文
