Transformers now runs llama.cpp quants

Hugging Face added GGUF support to transformers, letting users load Hub checkpoints sized for their machine via from_pretrained. To approach llama.cpp performance, it reuses ggml kernels and cuts generate overhead, initially targeting Apple Silicon and the Qwen3.5 architecture. Local tools like Ollama, LM Studio and Jan are powered by llama.cpp.

• Qwengram-0.8B: n-gram memory transfer cuts perplexity 5.05%2026-09-25
• Hugging Face releases VLM speculative decoding drafter2026-09-24
• Open-source engine Splash hits 144 tok/s on M5 Max2026-09-19
• Cactus releases Needle 3 automation model2026-09-18
• Voodoo Dynamic Quant goes MIT2026-09-15