Industry

Everything · newest first

Models, products and AI startups, near-duplicates collapsed to the most credible source

21M model with 6.4B lookup table matches 114M dense model, table on SSD

Developer fehcyy released sparse-memory-lm, giving a 21M-parameter model a 16.8M-row lookup table (6.4B table params, 33M used per token), achieving performance comparable to a 114M dense model on 500M Wikipedia tokens. The table can be 4-bit quantized and memory-mapped from NVMe SSD, achieving 140 tok/s on RX 9070 with only 0.4 GB VRAM, but long prompts are slow due to 4KB page misses. Triton kernels run unchanged on Radeon, MI350X, and H100/H200. Bolting a table onto a pretrained model didn't work.

r/LocalLLaMA ·

Comparing memory mechanisms across RNNs, Transformers and SSMs

A community post compares architectures through working memory: RNNs compress history into a recurrent state with roughly O(N) state versus O(N²) parameters, Transformers store past representations in a growing KV cache while weights stay frozen, and selective SSMs like Mamba return to fixed-size recurrent memory with input-dependent retention.

r/MachineLearning ·

User Buys Mining Rig With 54GB VRAM for $35

A user bought an old mining rig on a Russian classifieds site for 3,000 rubles (~$35) containing nine working P106 6GB GPUs totaling 54GB of VRAM. Only an SSD was missing; it booted fine from USB.

r/LocalLLaMA ·
Load more