ISTA-DASLab
6 storiesISTA-DASLab is a research lab focused on model quantization and low-bit inference, publishing GGUF quantizations built on its GSQ-RCO method on Hugging Face. Recent coverage includes a quantized DeepSeek-V4.1-Flash release, a prefiller approach for recovering low-quant accuracy on Qwen3.8-27B, and a 3.5 BPW Qwen-Flash-Next Coder quant running a 262K context on 8GB of VRAM. Unsloth has disputed the reproducibility of its gsq-rco benchmark numbers.
Related topics
ISTA-DASLab's gsq-rco quantization has drawn criticism from Unsloth for unreproducible benchmark numbers, with Unsloth finding its own quants outperform gsq-rco. The creator of the Basalt inference engine then replaced ISTA-DASLab-based quants with similar-sized Unsloth equivalents.
A user demonstrated running ISTA DASLab's 3.5 BPW Qwen-Flash-Next Coder quant on 8GB VRAM plus 24GB RAM, reaching 262K context with about 70 T/s prefill and 8 T/s generation via MoE offloading and disk n-gram. Half the experts are pruned, giving an effective 1.89 bits per parameter while retained weights stay at 3.5 BPW.
ISTA-DASLab published GGUF quantizations of DeepSeek-V4.1-Flash-1017 using its GSQ-RCO quantization method, available on Hugging Face. The model is not yet supported in llama.cpp because original architecture support has not been merged.
ISTA-DASLab demonstrated a prefiller model approach that recovers low-quant accuracy for Qwen3.8-27B while retaining low VRAM usage and fast decoding. It requires an Nvidia Blackwell GPU and shows no speed penalty for contexts longer than ~4k tokens.
A user reports that the Strata inference engine with ISTA-DASLab's quantized Qwen3.8-Flash-Next hits 51 t/s generation and 1500 t/s prompt processing on a 12GB VRAM, 64GB RAM laptop, far outpacing stock llama.cpp. Strata currently supports only Nvidia and select GGUF models.
A developer built a custom inference engine, Strata, running Qwen3.8-Flash-Next on consumer hardware including a 12GB RTX 5070, achieving about 65 tokens/s output and 430 tokens/s prompt processing, a major gain over the earlier llama.cpp setup. The engine is open-sourced, one-click installable, and currently optimized for CUDA only.
That is everything