WebGPU

7 stories

WebGPU is a browser graphics and compute API that lets JavaScript applications use the GPU for local inference. Recent coverage centers on running models in the browser: Hugging Face open-sourced WebGPU kernels for more than 200 ML operations and is upstreaming them, Ternary Bonsai 2's 2B model runs via custom WebGPU kernels at up to about 500 tok/s on an M4 Max, and LFM3-Omni-2B ran offline on an iPhone. CRAP, Talus and MicroLLM Lab also use WebGPU.

Related topics

Developer rebuilds Mozilla's Orbit with local inference

After Mozilla discontinued its AI browser summarizer Orbit, which sent data to GCP servers, a developer spent a month rebuilding it as Apogee, a fully local and privacy-focused extension. Apogee does not send or store user data, connects to a local Ollama instance for inference, and adds WebGPU for Chrome and Transformers.js for Firefox. It summarizes articles, YouTube videos, Wikipedia pages, and forum threads.

Hacker News ·
That is everything