Qwen3.5-0.8B
6 storiesQwen3.5-0.8B is a 0.8B-parameter small language model in the Qwen family. Recent discussion has focused on weight-level edits: a community post says MIT CSAIL used GPT-6 Astra to rewrite about 0.016% of its weights, fixing "Wait" reasoning loops and raising MATH-500 from 7.9% to 48.1%, with the diff released. A developer also transferred Qwen3.8 Flash-Next's PLE n-gram memory into the model, cutting validation perplexity from 18.2759 to 17.3534.
Related topics
Tether AI Research unveiled BotPsy, a 0.8B-parameter model built on Qwen3.5-0.8B for function calling and agentic workflows, deployable on-device. It claims state-of-the-art tool use for its size, beating every model up to 1.2B it tested, targeting low-to-medium tasks with data privacy, though not for high-stakes use.
Following MIT CSAIL's weight-space programming approach, a user had Claude Opus 5.5 rewrite 8 MLP neurons in Gemma 4 12B without training—about 92k of 11.9B parameters—to choose the Italian imperfect tense only in appropriate contexts.
A community post says MIT CSAIL fixed Qwen3.5-0.8B's reasoning loops by editing 0.016% of its weights using GPT-6 Astra, raising MATH-500 from 7.9% to 48.1%, and released the diff. The poster applied Anthropic's Jacobian lens and found the model reads 'Wrong' signals when repeating 'Wait', replaced by 'Verification' after the patch.
A study introduces "Weight-Space Programming": GPT-6 Astra can infer a network's training goal from raw weights alone and, without environment feedback, write small game-playing networks from scratch that beat PPO-trained models trained for 100 million steps. Astra also located weight issues causing reasoning loops in Qwen3.5-0.8B, editing just 0.016% of weights to lift MATH-500 from 7.9% to 48.1% and triple IFEval.
Developer fehcyy released sparse-memory-lm, giving a 21M-parameter model a 16.8M-row lookup table (6.4B table params, 33M used per token), achieving performance comparable to a 114M dense model on 500M Wikipedia tokens. The table can be 4-bit quantized and memory-mapped from NVMe SSD, achieving 140 tok/s on RX 9070 with only 0.4 GB VRAM, but long prompts are slow due to 4KB page misses. Triton kernels run unchanged on Radeon, MI350X, and H100/H200. Bolting a table onto a pretrained model didn't work.
A developer transferred Qwen3.8 Flash-Next's pretrained PLE n-gram memory into Qwen3.5-0.8B, keeping the backbone and ~51B-parameter memory frozen while training an R=1 reader; validation perplexity fell from 18.2759 to 17.3534, a 5.05% reduction. This is a language-model validation result, not a 5% benchmark accuracy gain.
That is everything