On September 1, Hugging Face announced 207 versioned WebGPU kernels and Fleet, an in-browser test suite intended to collect correctness and performance evidence across real devices. The announcement reports a 2.57x geometric-mean speedup and 1.90x median speedup against ONNX Runtime Web on an Apple M4 GPU, after retaining 809 comparable cases from 1,756 tests. It also says those timings cover GPU work rather than setup, model loading, shader compilation, or output readback. They are vendor-reported measurements, not a result for every browser, model, or machine.

The timing matters because “runs locally” is becoming too blunt a description. A browser may download a model and runtime, execute some work on a GPU, fall back to WebAssembly when WebGPU is unavailable, and still send telemetry or requests to a server elsewhere in the application. Transformers.js documents WebGPU as an experimental path, with support and behavior varying by browser and device.

The reader question is therefore narrower and more useful: what evidence is needed before a small model can be accepted as a browser-local component of a real workflow? This is a decision framework, not a hands-on speed review. No independent model, browser matrix, privacy audit, or accuracy result is claimed.

Card 1 — Define “local” as a data path

Draw the path before measuring the model. Separate package discovery, model-weight download, cache reuse, inference, error reporting, benchmark submission, fallback, and update. Label each step local, remote, optional, or unknown.

The model can execute locally while the first run fetches weights from a repository. A browser fallback can preserve the user experience while moving inference to a different runtime or server. A consented benchmark can improve a kernel project while still creating a distinct data transfer that must be disclosed.

Write the privacy claim at the same granularity. “The prompt never leaves the device” is a testable claim about inference traffic, not a blanket claim about downloads, diagnostics, accounts, or update checks. Record the behavior when WebGPU is missing, the cache is empty, the model fails to load, or the user revokes consent.

Card 2 — Freeze the model, conversion, and license

A model name is not a reproducible artifact. Record the repository, full revision hash, configuration, tokenizer, browser format, quantization, runtime version, kernel package, and hash of the files actually shipped. The Hugging Face download guide says the latest main revision is used by default and supports downloading at a tag, branch, or full commit hash. Use the full hash for a test and keep the manifest.

The Qwen3.5-0.8B model card is a useful example of the distinction. It identifies a 0.8B model, labels the repository Apache-2.0, lists compatible server runtimes, and describes intended use as prototyping, task-specific fine-tuning, and research or development. Those are publisher-provided facts and claims. They do not establish that a browser conversion preserves the same behavior, that the license covers every surrounding asset, or that the model is suitable for a production task.

Treat the license and the browser artifact as separate review cards. A permissive model license does not make an output accurate, private, or fit for a sensitive workflow.

Card 3 — Build a device matrix, not a hero benchmark

Use rows for browser and version, operating system, GPU, driver, memory, power state, and runtime path. Use columns for cold download, cache hit, initialization, first output, steady-state latency, errors, fallback behavior, and task quality. Include at least one supported WebGPU path, one lower-capability device, and one forced fallback.

The Transformers.js WebGPU guide reports global WebGPU support at around 85% as of March 2026 and warns that non-Chromium behavior can be experimental. That is not a fleet result for a particular application. The Hugging Face kernel announcement likewise explains that performance changes with the accelerator, browser, driver, input shape, and available features.

Keep the benchmark’s unit visible. A kernel timing is not a model latency; a model latency is not an end-to-end user wait. Do not transfer an Apple M4 comparison to a mixed laptop population without measuring the missing rows.

Card 4 — Count complete operating cost

Report more than tokens per second. Measure download bytes, cache hit rate, load and compile time, peak memory, warm and cold latency, error rate, battery or thermal impact where relevant, and the cost of a server fallback. For the task itself, record exact correctness, omissions, unsupported additions, and reviewer correction time. Keep those quality measures separate from speed.

The September 1 announcement explicitly excludes setup, kernel loading, session creation, input upload, and output readback from its operation-level timings. That makes its comparison interpretable, not useless. It also tells an application team what remains to be measured.

Use a small synthetic or de-identified fixture with a known answer key. Repeat cold and warm runs. Record shader compilation, downloads, memory sharing with other tabs, and fallback. A model that is fast after a long first-run setup may still be the wrong fit for a short-lived workflow.

Card 5 — Set the failure and update gate

Decide the acceptance rule before looking at the chart. An example gate might require: no sensitive prompt may cross the declared boundary; every supported device class must meet a task-specific correctness floor; cold start and warm latency must fit the workflow; fallback must be explicit; and the shipped artifact must be reproducible from a pinned manifest.

Test the failure states deliberately: WebGPU unavailable, permissions denied, driver error, interrupted download, stale cache, missing kernel, model and runtime mismatch, memory pressure, browser update, and network loss after the initial load. Record whether the system refuses, retries, falls back, or silently changes path.

Versioned kernel repositories are promising because they can carry contracts, correctness cases, benchmark cases, and digests alongside an implementation. That structure helps, but it does not remove the application’s obligation to pin versions and replay its own task fixture. A changed model conversion or kernel selection is a new arm of the test.

The decision should name the operating point, not crown a universal winner: accept this model, artifact, runtime, and device class for this task under this data path. Hold when a fallback, privacy claim, or quality result is unknown. The practical output is a release record that another operator can reproduce, not the phrase “AI runs in the browser.”

Sources and limitations

  1. Hugging Face, “Introducing @huggingface/kernels: 200+ WebGPU Kernels for Local AI” — checked September 14, 2026; supports the September 1 release, versioned kernel structure, Fleet, the reported Apple M4 comparison, and the vendor’s timing limitations. Limitation: this is the publisher’s announcement and benchmark; it does not establish model-level or application-level performance on another device.
  2. Hugging Face Transformers.js, “Running models on WebGPU” — checked September 14, 2026; supports browser-side WebGPU execution, the documented support estimate, device and browser caveats, and fallback concerns. Limitation: documentation is not a test of the proposed workflow.
  3. Qwen, “Qwen3.5-0.8B” model card — checked September 14, 2026; supports the model-card metadata, parameter label, Apache-2.0 label, stated intended uses, and listed runtime integrations. Limitation: publisher documentation does not prove browser compatibility, task quality, privacy, or legal coverage of surrounding assets.
  4. Hugging Face Hub, “Download files from the Hub” — checked September 14, 2026; supports revision pinning, full commit hashes, repository snapshots, and local caching behavior. Limitation: download documentation does not audit an application’s supply chain or network behavior.

No model or browser workflow was run for this draft. No independent benchmark, privacy audit, licensing opinion, or production recommendation is claimed. The proposed device matrix, fixture, gates, and cost ledger must be executed against a pinned artifact before a team calls a browser-local model ready.