The A.I. Daily
Developer proposes speculative decoding using 2-bit draft plus RAM residual
A developer published a Hugging Face repo that keeps a 2-bit draft model in VRAM and a 6-8 bit residual in system RAM, sending only activations over PCIe so the draft costs no extra memory. Tests show a 6-bit residual nears Q8_0 quality while reading about 28% fewer bytes.