Hugging Face 发布 @huggingface/kernels：207 个 WebGPU kernel 支持本地 AI

TL;DR

207 WebGPU kernels , published as individual repositories in the webgpu-kernels organization. Apache-2.0 licensed.

A JavaScript loader , @huggingface/kernels , which downloads, prepares, and runs kernels directly from the Hub.

Explicit contracts and reproducible evidence for every kernel, including manifests, correctness tests, benchmark cases, and WGSL shader templates.

Fleet , a browser-based benchmarking tool that crowdsources correctness and performance evidence across real-world GPUs to help us improve kernels and their variants.

Why start with kernels?

A model running in the browser eventually becomes a sequence of GPU operations: matrix multiplications, normalizations, convolutions, attention primitives, quantization operations, data-layout transformations, and many more. WebGPU makes these operations available across modern browsers through a portable API, while WGSL provides a common language for the shaders that execute them.

Portability, however, does not automatically mean performance. Two shaders can implement the same operation and produce the same output while behaving completely differently across different accelerators. Workgroup sizes, memory access patterns, vectorization, data types, and fusion strategies can all affect performance. The best choice can also change with the input shape, device, browser, and available WebGPU features.

This is why kernels form a foundational layer of fast browser inference. Higher-level runtimes can only be as efficient as the operations they dispatch. By making those operations individually discoverable, testable, benchmarkable, and versioned, we can improve the foundation independently while keeping a stable contract for the layers above it.

A kernel repository, not just a shader

Each kernel in the collection has its own repository and kernel card. The card documents the operation's semantics, inputs, outputs, attributes, supported data types, source files, and a ready-to-run @huggingface/kernels example.

For example, ai.onnx.Add implements elementwise addition with multidirectional broadcasting. It is one of the simplest operations in a neural network, used everywhere from residual connections to adding a bias. Its card documents the two inputs, the broadcasted output shape, supported data types, and the variants available for different shapes and devices.

Behind the card, the repository contains the artifacts needed to understand and evaluate the implementation:

manifest.json is the source of truth for the operation contract. It defines inputs, outputs, attributes, type constraints, and shape derivation rules.

metadata.json records the kernel identifier, digests, and provenance.

test.json contains correctness cases, so an implementation can be checked against expected behavior.

bench.json contains benchmark and tuning cases that represent the workloads used to evaluate the kernel.

*.wgsl.jinja files contain the parameterized WGSL implementations used to produce shaders for a particular request and device.

This structure turns a shader into a reusable software artifact. The interface is inspectable without reading WGSL, correctness and performance cases travel with the implementation, and published versions can be loaded explicitly rather than depending on an unversioned file URL. Our kernels can also serve as reference implementations for developers building custom WebGPU kernels or integrating these operations into their own runtimes.

Loading a kernel from the Hub

Install the package from npm:

npm install @huggingface/kernels@preview

Running these kernels requires a browser with WebGPU support . WebGPU availability depends on the browser, operating system, GPU, and driver. You can check for it in JavaScript with "gpu" in navigator .

@huggingface/kernels provides the bridge between a kernel repository and your application. Call getKernel with a Hub repository ID and a contract version, then invoke the returned function with typed input data and tensor shapes. Here is a small bias-add example:

import { getKernel } from "@huggingface/kernels" ;

const add = await getKernel ( "webgpu-kernels/ai.onnx.Add" , { version : 1 });

const { c } = await add ({ a : { data : new Float32Array ([ 1 , 2 , 3 , 4 , 5 , 6 ]), shape : [ 2 , 3 ], }, b : { data : new Float32Array ([ 10 , 20 , 30 ]), shape : [ 3 ], }, });

The second input is broadcast across the first dimension, producing an output with shape [2, 3] . The loader derives that output shape and logical data type from the manifest contract and the inputs, then allocates c automatically.

Addition on six floats is deliberately the smallest possible demo. At this size, the GPU round trip costs far more than the math. The point is the call pattern: it stays exactly the same for the heavyweight operations where optimized kernels actually pay off, such as matrix multiplication ( ai.onnx.MatMul ). Only the repository ID and the inputs change.

Even this elementary operation illustrates why kernels need variants. Equal-shape addition can use a direct vectorized path, while broadcasted inputs need different indexing logic. The published Add kernel includes variants for equal shapes, vectorized broadcasting, scalar processing, and general broadcasting. The runtime can select an implementation that fits the current call and device without changing the application-facing API.

The version: 1 option selects version 1 of the published kernel contract . It is separate from an ONNX opset, an operator's since_version , or a model revision. Keeping those concepts separate lets applications depend on a stable JavaScript-facing contract while kernel implementations evolve behind it.

How fast are the kernels?

So, how much of a difference do optimized kernels actually make? We put our collection head-to-head with ORT WebGPU on an Apple M4 GPU, using ONNX Runtime Web 1.30.0-dev.20260826-b1f76d586a . We started with 1,756 test cases across all 207 operations and kept the 809 cases where both sides produced matching outputs and reliable timings.

Across those comparisons, our kernels were 2.57x faster by geometric mean and 1.90x faster at the median , with 629 wins, 176 losses, and 4 ties. Here is a closer look at four familiar operations:

Some individual wins were much bigger. A particularly difficult bilinear Einsum case ( i,ij,j with size 4096) ran in 0.136 ms with our kernel versus 1,396 ms with ORT WebGPU: more than 10,000x faster . A row-wise CumSum over [256, 4096] was 301x faster , at 0.016 ms versus 4.784 ms. These are unusual cases rather than the speedups you should expect everywhere, but they show how much a specialized kernel can help when a general implementation hits a slow path.

We timed the work done on the GPU itself, leaving out setup such as loading kernels, creating sessions, uploading inputs, compiling shaders, and reading outputs back. Very short workloads are naturally harder to measure, and small cases can benefit from the GPU cache, so these numbers are best read as a useful comparison rather than a promise for every application.

They are also results for individual operations, not complete models. Exact performance will change across GPUs and browsers, which is why Fleet is so important for building a broader picture.

We are also working with the ONNX Runtime team to upstream these improvements so they can benefit the broader ONNX Runtime Web ecosystem.

From one device to a fleet

WebGPU performance varies across GPUs, browsers, and drivers, so results from one machine only tell part of the story. Fleet lets anyone run correctness and performance checks in the browser and see how the kernels behave on their hardware.

With consent, each run privately contributes evidence that helps us spot device-specific failures, compare variants, and improve selection rules. The goal is simple: use broad, real-world coverage to make the kernels faster and more reliable for everyone.

Building a shared foundation for WebAI

The initial 207 kernels are a starting point, not the end state. Publishing kernels independently on the Hub gives us a common place to inspect contracts, compare implementations, reproduce correctness checks, and improve performance without embedding every shader directly into every runtime.

The collection is also part of the Hub's broader kernel ecosystem: on the Kernels page , the WebGPU kernels sit alongside kernels for CUDA, ROCm, Metal, and other platforms, and can be filtered, sorted, and explored like any other artifact on the Hub.

The pieces reinforce one another:

Kernel repositories define transparent, versioned operation contracts.

@huggingface/kernels makes those operations straightforward to load and run from JavaScript.

Fleet crowdsources real-world evidence across a much broader range of devices than a conventional benchmark lab can cover.

Every contributed run can reveal failures, guide tuning, improve variant selection, and help validate future kernel versions.

This is the low-level foundation for the next steps in our browser inference stack. We are excited to connect these kernels to higher-level model tooling, continue expanding operation coverage, and make fast local inference easier to use across the WebAI ecosystem.

Explore the WebGPU kernel collection , try @huggingface/kernels , and join the Fleet to contribute evidence from your device and help us make the kernels better for everyone.

More Articles from our Blog

Async GRPO with LoRA across HF Jobs: a bucket, a proxy, and no NCCL

53 September 10, 2026 announcement multimodal transformers Welcome Inkling by Thinking Machines +1 167 July 15, 2026 Community 1 \n","updatedAt":"2026-09-01T17:02:12.555Z","author":{"_id":"6909cd87391d11cef04e07de","avatarUrl":"/avatars/d51475ad4be08357269e675c98c0673b.svg","fullname":"马保国","name":"mabaoguo00","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.18271753191947937},"editors":["mabaoguo00"],"editorAvatarUrls":["/avatars/d51475ad4be08357269e675c98c0673b.svg"],"reactions":[],"isReport":false}},{"id":"6a981b3abab49b84bccb227d","author":{"_id":"62f32eab52ad88c930bb3f3b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1677134945205-62f32eab52ad88c930bb3f3b.png","fullname":"Asankhaya Sharma","name":"codelion","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":452,"isUserFollowing":false,"primaryOrg":{"avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/680ff4388f704be391757780/-9cmzdDMCol0OfkTYK8jF.png","fullname":"Hugging Science","name":"hugging-science","type":"org","isHf":false}},"createdAt":"2026-09-02T12:48:58.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"The Fleet part may matter as much as the kernels. The 809 comparable cases on an M4 are useful, but browser, driver and device variance is where WebGPU tends to get painful. Will Fleet eventually expose per-device distributions, or is the plan to use the runs only for aggregate selection rules?","html":" The Fleet part may matter as much as the kernels. The 809 comparable cases on an M4 are useful, but browser, driver and device variance is where WebGPU tends to get painful. Will Fleet eventually expose per-device distributions, or is the plan to use the runs only for aggregate selection rules? \n","updatedAt":"2026-09-02T12:48:58.079Z","author":{"_id":"62f32eab52ad88c930bb3f3b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1677134945205-62f32eab52ad88c930bb3f3b.png","fullname":"Asankhaya Sharma","name":"codelion","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":452,"isUserFollowing":false,"primaryOrg":{"avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/680ff4388f704be391757780/-9cmzdDMCol0OfkTYK8jF.png","fullname":"Hugging Science","name":"hugging-science","type":"org","isHf":false}}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.964321494102478},"editors":["codelion"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1677134945205-62f32eab52ad88c930bb3f3b.png"],"reactions":[],"isReport":false},"replies":[{"id":"6a9af811abd119dc9011de67","author":{"_id":"63e3004bc123238b189e8c7a","avatarUrl":"/avatars/cb3536395bbf8c020c624d8953443890.svg","fullname":"Steven Roussey","name":"sroussey","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":0,"isUserFollowing":false},"createdAt":"2026-09-04T16:55:45.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"In an automated build system, I could see per device kernels, made more interesting by fusion of ops.\n\nThe area that obviously needs more attention are the 947 tests that did not result in reliable output. This could be the test, it could be that the ORT WebGPU is wrong, or the new kernel is wrong, or the test harness had issues, hard to know from the outside. Perhaps the Fleet thing is meant to improve this.\n\nVery excited to see such attention at making WebGPU first class in ML!","html":" In an automated build system, I could see per device kernels, made more interesting by fusion of ops. \n The area that obviously needs more attention are the 947 tests that did not result in reliable output. This could be the test, it could be that the ORT WebGPU is wrong, or the new kernel is wrong, or the test harness had issues, hard to know from the outside. Perhaps the Fleet thing is meant to improve this. \n Very excited to see such attention at making WebGPU first class in ML! \n","updatedAt":"2026-09-04T16:55:45.939Z","author":{"_id":"63e3004bc123238b189e8c7a","avatarUrl":"/avatars/cb3536395bbf8c020c624d8953443890.svg","fullname":"Steven Roussey","name":"sroussey","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":0,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9802036285400391},"editors":["sroussey"],"editorAvatarUrls":["/avatars/cb3536395bbf8c020c624d8953443890.svg"],"reactions":[],"isReport":false,"parentCommentId":"6a981b3abab49b84bccb227d"}}]},{"id":"6a9b113fc910ff7e484860bb","author":{"_id":"62f32eab52ad88c930bb3f3b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1677134945205-62f32eab52ad88c930bb3f3b.png","fullname":"Asankhaya Sharma","name":"codelion","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":452,"isUserFollowing":false,"primaryOrg":{"avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/680ff4388f704be391757780/-9cmzdDMCol0OfkTYK8jF.png","fullname":"Hugging Science","name":"hugging-science","type":"org","isHf":false}},"createdAt":"2026-09-04T18:43:11.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"@sroussey Agreed—the excluded 947 need a taxonomy before the 2.57× summary can travel. I'd want counts by operation and device for unsupported, ORT mismatch, new-kernel mismatch, and harness/timeout, with raw failing cases downloadable. Then Fleet can optimize only within a proven-correct compatibility envelope; fusion and per-device tuning become much safer.","html":" @ sroussey Agreed—the excluded 947 need a taxonomy before the 2.57× summary can travel. I'd want counts by operation and device for unsupported, ORT mismatch, new-kernel mismatch, and harness/timeout, with raw failing cases downloadable. Then Fleet can optimize only within a proven-correct compatibility envelope; fusion and per-device tuning become much safer. \n","updatedAt":"2026-09-04T18:43:11.912Z","author":{"_id":"62f32eab52ad88c930bb3f3b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1677134945205-62f32eab52ad88c930bb3f3b.png","fullname":"Asankhaya Sharma","name":"codelion","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":452,"isUserFollowing":false,"primaryOrg":{"avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/680ff4388f704be391757780/-9cmzdDMCol0OfkTYK8jF.png","fullname":"Hugging Science","name":"hugging-science","type":"org","isHf":false}}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9059944748878479},"editors":["codelion"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1677134945205-62f32eab52ad88c930bb3f3b.png"],"reactions":[],"isReport":false}},{"id":"6a9bf1c96372c5bf8d74ff27","author":{"_id":"6a9bb677f1430dc1c9f2d9f2","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6a9bb677f1430dc1c9f2d9f2/kU3z1y_9g6WfvF_w65q6i.png","fullname":"Alvaro Fernandez","name":"arubaro75","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false},"createdAt":"2026-09-05T10:41:13.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Good Job.","html":" Good Job. \n","updatedAt":"2026-09-05T10:41:13.525Z","author":{"_id":"6a9bb677f1430dc1c9f2d9f2","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6a9bb677f1430dc1c9f2d9f2/kU3z1y_9g6WfvF_w65q6i.png","fullname":"Alvaro Fernandez","name":"arubaro75","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.24303191900253296},"editors":["arubaro75"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/6a9bb677f1430dc1c9f2d9f2/kU3z1y_9g6WfvF_w65q6i.png"],"reactions":[{"reaction":"❤️","users":["nico-martin","Xenova"],"count":2}],"isReport":false}},{"id":"6ab518ba3f74967dc4d28850","author":{"_id":"6408f745bf3e9a4bb2e163c1","avatarUrl":"/avatars/1d55c6d2d46e78f2d32bc528f5efb1da.svg","fullname":"Milko Madariaga","name":"mmadaria","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2026-09-24T12:34:02.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Excellent! This will help our WebAI community continue developing highly optimized solutions. Many thanks @Xenova and @nico-martin for the great work you have been doing.","html":" Excellent! This will help our WebAI community continue developing highly optimized solutions. Many thanks @ Xenova and @ nico-martin for the great work you have been doing. \n","updatedAt":"2026-09-24T12:34:02.388Z","author":{"_id":"6408f745bf3e9a4bb2e163c1","avatarUrl":"/avatars/1d55c6d2d46e78f2d32bc528f5efb1da.svg","fullname":"Milko Madariaga","name":"mmadaria","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9323468208312988},"editors":["mmadaria"],"editorAvatarUrls":["/avatars/1d55c6d2d46e78f2d32bc528f5efb1da.svg"],"reactions":[{"reaction":"❤️","users":["Xenova","nico-martin"],"count":2}],"isReport":false}}],"status":"open","isReport":false,"pinned":false,"locked":false,"collection":"community_blogs"},"contextAuthors":["nico-martin","Xenova"],"primaryEmailConfirmed":false,"discussionRole":0,"acceptLanguages":["en","zh"],"withThread":true,"cardDisplay":false,"repoDiscussionsLocked":false,"hideComments":true}"> mabaoguo00 30 days ago 1

Reply codelion 29 days ago The Fleet part may matter as much as the kernels. The 809 comparable cases on an M4 are useful, but browser, driver and device variance is where WebGPU tends to get painful. Will Fleet eventually expose per-device distributions, or is the plan to use the runs only for aggregate selection rules?

1 reply

sroussey 27 days ago In an automated build system, I could see per device kernels, made more interesting by fusion of ops.

The area that obviously needs more attention are the 947 tests that did not result in reliable output. This could be the test, it could be that the ORT WebGPU is wrong, or the new kernel is wrong, or the test harness had issues, hard to know from the outside. Perhaps the Fleet thing is meant to improve this.

Very excited to see such attention at making WebGPU first class in ML!

codelion 27 days ago @ sroussey Agreed—the excluded 947 need a taxonomy before the 2.57× summary can travel. I'd want counts by operation and device for unsupported, ORT mismatch, new-kernel mismatch, and harness/timeout, with raw failing cases downloadable. Then Fleet can optimize only within a proven-correct compatibility envelope; fusion and per-device tuning become much safer.

Reply arubaro75 26 days ago Good Job.

Reply mmadaria 7 days ago Excellent! This will help our WebAI community continue developing highly optimized solutions. Many thanks @ Xenova and @ nico-martin for the great work you have been doing.

Reply Edit Preview Upload images, audio, and videos by dragging in the text input, pasting, or clicking here . Tap or paste here to upload images Comment · Sign up or log in to comment

+78