21M model with 6.4B lookup table matches 114M dense model, table on SSD

Developer fehcyy released sparse-memory-lm, giving a 21M-parameter model a 16.8M-row lookup table (6.4B table params, 33M used per token), achieving performance comparable to a 114M dense model on 500M Wikipedia tokens. The table can be 4-bit quantized and memory-mapped from NVMe SSD, achieving 140 tok/s on RX 9070 with only 0.4 GB VRAM, but long prompts are slow due to 4KB page misses. Triton kernels run unchanged on Radeon, MI350X, and H100/H200. Bolting a table onto a pretrained model didn't work.

• Ai2 replaces priority scheduler with GPU time budgets2026-10-09
• CoWindow and MassAlloc Attention Speed Up Long-Context Models2026-09-30
• Jeeves: Reasoning Improves Jev-Like Decision Models2026-09-29
• Buy vs Rent an H200 Box: Break-Even Math Puts Tipping Point at 60% Utilization2026-09-25
• Qwengram-0.8B: n-gram memory transfer cuts perplexity 5.05%2026-09-25