Developer releases Ninfer 4080 for 16GB GPUs with 100k context

A developer released Ninfer 4080, an inference solution that runs ISTA-DASLab-Qwen-3.8-27B-GSQ at 100k context on a 16GB RTX 4080, peaking at 2720 tok/s prefill and 262 tok/s generation. The project is open-sourced to bring better performance to 16GB-class GPUs than general-purpose engines.

• Four RTX 3060 Ti GPUs Hit 120 t/s Local Inference With 262K Context2026-09-27
• llama.cpp v0.6.0 Released With MTP Decoding2026-10-05
• Developer Runs Qwen3.5 LLMs on Cheap FPGA Mining Hardware2026-10-04
• llama.cpp Outperforms Alternative Engine on Long-Form Reasoning2026-10-04
• User Shares Strata Speedup for Local LLMs but 64GB RAM Constraint2026-10-04