Open-source research proposes new sparse attention for ultra-long context

An open-source research team shows attention can be sparsified on Qwen 3.8 27B without observed accuracy loss, and that extending context to 10M tokens hurts some tasks but leaves most, like NIAH, fine. It says sparse attention can be trained natively on next-token loss, matching full-attention validation loss faster; kernels, SGLang integration and the paper are open.

• Study probes Qwen's 'Wait' reasoning loops2026-11-04
• User mines 16 years of city council notes with local Qwen model2026-11-03
• Alibaba Qwen Code Repo Reportedly Leaks qwen4-max and Other Model IDs2026-11-03
• User says Qwen 3.8 27b used Blender to do math2026-11-03
• Study questions LLMs' ability to read neural net weights2026-11-03