User builds 256GB VRAM rig with 8 Radeon Pro V620s

A user built a roughly $2,800 inference rig with eight used Radeon Pro V620 cards totaling 256GB VRAM, running Qwen3.8-Flash-Next via a custom vLLM fork. It reached 60-100 tokens/s decode and over 3,000 tokens/s prefill, roughly 8x faster than llama.cpp.

• Developer releases Ninfer 4080 for 16GB GPUs with 100k context2026-10-04
• User Shares Strata Speedup for Local LLMs but 64GB RAM Constraint2026-10-04
• Two 96GB Ascend Cards Run Qwen3.8 Flash-Next2026-10-02
• llama.cpp Adds GLM-5.3-Flash Support2026-09-30
• Four RTX 3060 Ti GPUs Hit 120 t/s Local Inference With 262K Context2026-09-27