Hugging Face releases tokenizers v1 with big speed gains

Hugging Face released a v1 release candidate of tokenizers focused on performance, saying it is often tens of times faster than v0.23. The team says tokenization can bottleneck large-scale training, high-concurrency serving and long inputs, leaving GPUs idle. IBM, NVIDIA and the ExecuTorch team contributed patches and cross-hardware testing.

• Developer builds Halloween haunted house with NVIDIA Nemotron models2026-10-30
• Hugging Face launches AnySpeechRecognizer and AnyDecisionModel for Apple platforms2026-10-29
• DOJ queries Nvidia's $13B Hugging Face deal2026-10-28
• NVIDIA releases Nemotron 3.5 Transcribe model and Granary v2 pipeline2026-10-28
• Nvidia CEO demos Microduck robots, preorders exceed 20,0002026-10-23