WebGPU
7 storiesWebGPU is a browser graphics and compute API that lets JavaScript applications use the GPU for local inference. Recent coverage centers on running models in the browser: Hugging Face open-sourced WebGPU kernels for more than 200 ML operations and is upstreaming them, Ternary Bonsai 2's 2B model runs via custom WebGPU kernels at up to about 500 tok/s on an M4 Max, and LFM3-Omni-2B ran offline on an iPhone. CRAP, Talus and MicroLLM Lab also use WebGPU.
Related topics
A Reddit user reports running LFM3-Omni-2B fully offline on an iPhone 18 Pro Max at about 80 tokens per second using custom WebGPU kernels in a React Native app, highlighting on-device multimodal inference potential.
CRAP, a Gemma 4 E2B fine-tune, skips the compiler and turns prompts directly into runnable Linux x86_64 ELF binaries. v1.1 scores 26/40 on novel strings (v1: 8/40) and 10/12 on multi-turn (v1: 0/12). A WebGPU browser demo runs the ~3.6 GiB Q6_K quantization, scoring 109/144.
Prism ML released the Ternary Bonsai 2 small model series (2B, 4B, 9B) on Hugging Face. The 2B runs fully locally in the browser via custom WebGPU kernels, reaching up to about 500 tok/s on an M4 Max, with a WebGPU demo provided.
Talus is a 23M-parameter diffusion model trained on 45,000 procedurally generated terrain maps to generate 64x64 heightmaps from terrain type and five properties. Evaluated against a real-vs-real noise floor, it runs in the browser via WebGPU.
After Mozilla discontinued its AI browser summarizer Orbit, which sent data to GCP servers, a developer spent a month rebuilding it as Apogee, a fully local and privacy-focused extension. Apogee does not send or store user data, connects to a local Ollama instance for inference, and adds WebGPU for Chrome and Transformers.js for Firefox. It summarizes articles, YouTube videos, Wikipedia pages, and forum threads.
Hugging Face open-sourced WebGPU kernels for over 200 common ML operations that run entirely locally in browsers. The optimizations are being upstreamed to Transformers.js, ONNX Runtime Web, LiteRT.js, and more.
MicroLLM Lab lets users run seven tiny LLMs in the browser using WebGPU and IndexedDB locally. It displays speed and accuracy metrics and can generate a verifiable performance certificate.
That is everything