Qwen3.5-0.8B

6 stories

Qwen3.5-0.8B is a 0.8B-parameter small language model in the Qwen family. Recent discussion has focused on weight-level edits: a community post says MIT CSAIL used GPT-6 Astra to rewrite about 0.016% of its weights, fixing "Wait" reasoning loops and raising MATH-500 from 7.9% to 48.1%, with the diff released. A developer also transferred Qwen3.8 Flash-Next's PLE n-gram memory into the model, cutting validation perplexity from 18.2759 to 17.3534.

Related topics

Study probes Qwen's 'Wait' reasoning loops

A community post says MIT CSAIL fixed Qwen3.5-0.8B's reasoning loops by editing 0.016% of its weights using GPT-6 Astra, raising MATH-500 from 7.9% to 48.1%, and released the diff. The poster applied Anthropic's Jacobian lens and found the model reads 'Wrong' signals when repeating 'Wait', replaced by 'Verification' after the patch.

r/LocalLLaMA ·

Astra Natively Reads and Writes Weights to Improve Qwen

A study introduces "Weight-Space Programming": GPT-6 Astra can infer a network's training goal from raw weights alone and, without environment feedback, write small game-playing networks from scratch that beat PPO-trained models trained for 100 million steps. Astra also located weight issues causing reasoning loops in Qwen3.5-0.8B, editing just 0.016% of weights to lift MATH-500 from 7.9% to 48.1% and triple IFEval.

Hacker News ·

21M model with 6.4B lookup table matches 114M dense model, table on SSD

Developer fehcyy released sparse-memory-lm, giving a 21M-parameter model a 16.8M-row lookup table (6.4B table params, 33M used per token), achieving performance comparable to a 114M dense model on 500M Wikipedia tokens. The table can be 4-bit quantized and memory-mapped from NVMe SSD, achieving 140 tok/s on RX 9070 with only 0.4 GB VRAM, but long prompts are slow due to 4KB page misses. Triton kernels run unchanged on Radeon, MI350X, and H100/H200. Bolting a table onto a pretrained model didn't work.

r/LocalLLaMA ·
That is everything