The A.I. Daily
21M model with 6.4B lookup table matches 114M dense model, table on SSD
Developer fehcyy released sparse-memory-lm, giving a 21M-parameter model a 16.8M-row lookup table (6.4B table params, 33M used per token), achieving performance comparable to a 114M dense model on 500M Wikipedia tokens. The table can be 4-bit quantized and memory-mapped from NVMe SSD, achieving 140 tok/s on RX 9070 with only 0.4 GB VRAM, but long prompts are slow due to 4KB page misses. Triton kernels run unchanged on Radeon, MI350X, and H100/H200. Bolting a table onto a pretrained model didn't work.