8GB GPU runs 262K-context Qwen quant at 70 T/s

A user demonstrated running ISTA DASLab's 3.5 BPW Qwen-Flash-Next Coder quant on 8GB VRAM plus 24GB RAM, reaching 262K context with about 70 T/s prefill and 8 T/s generation via MoE offloading and disk n-gram. Half the experts are pruned, giving an effective 1.89 bits per parameter while retained weights stay at 3.5 BPW.

• ISTA-DASLab releases quantized DeepSeek-V4.1-Flash2026-10-24
• llama.cpp PR cuts Qwen Flash Next VRAM use2026-10-03
• llama.cpp merges MTP support for Qwen Flash Next2026-10-01
• Qwen3.8 runs locally at 51 t/s on 12GB laptop GPU2026-09-30
• Qwen3.8-Flash-Next Hits 65 Tokens/Second on 12GB VRAM2026-09-24