Issue 2026-11-11 · Industry · 开源

llama.cpp Adds FP8 KV-Cache Support

Open-source inference framework llama.cpp has merged a change adding FP8 support for the KV-cache, a move expected to reduce memory usage and improve efficiency for large model inference.

r/LocalLLaMA5 d ago
Read original ↗