Qwen3.8-Flash-Next is an open-weight Qwen model built as a mixture-of-experts, with roughly 180B total and 6B active parameters; a 125B variant also circulates. Recent coverage centers on local inference and quantization: Nvidia's 102GB FP4 quant, Mesh Applied's claim of 184 tok/s on one RTX 6000 Pro after NVFP4 quantization, the Basalt engine's speedup claims, and Lean-QwenFlash running the 125B model on 16GB VRAM plus 8GB RAM.
DFlash2 drafter lifts Qwen3.8-Flash-Next speed by 17%A user's tests show enabling the DFlash2 drafter for Qwen3.8-Flash-Next in llama.cpp v0.8.0 raises decode speed by about 17% geometric mean: +18% on code, +39% on lists, -3% on prose. The DFlash2 draft head is only ~350MB versus 2.2GB for MTP and supports vision input.
Qwen3.8-Flash-Next hits 184 tok/s on one RTX 6000 ProMesh Applied released an NVFP4-quantized Qwen3.8-27B-NInfer model, claiming 184 tok/s on a single RTX 6000 Pro. The team argues open models already suffice for about 98% of real AI tasks, so the bottleneck is running them often and fast, and it optimizes quantization and kernels around time per task.
Community discusses Qwen4 expectations and model size needsLocalLLaMA users discuss expectations for Qwen4, noting Qwen3.8-Flash-Next is impressive as a preview model and questioning whether Qwen4 will significantly surpass it or if the current model is already near the optimum for its size.
Lean-QwenFlash Runs 125B Model on Low MemoryA developer adapted the glm-flash-lite SSD offloading approach to Qwen3.8-Flash-Next (125B), creating Lean-QwenFlash, which decodes at ~20 tok/s with 16GB VRAM and 8GB RAM. An RTX 4090 hits 20 tok/s decode and 1,627 tok/s prefill at 262k context; an RTX 3080 laptop achieves 18-21 tok/s at 100k context. Code and scripts are open-sourced on GitHub.
Five Qwen3.8-27B models match Fable 5 in benchmark testsA paper and code repository introduce ledger-based zero-shot self-orchestration, in which fresh instances of one model split problems and coordinate through a shared filesystem. On the 100 latest hard LiveCodeBench problems, gains reach 25.6 points on pinned backends; orchestrated Qwen3.8 Flash Next hits 93.0% pass@1 versus Fable 5's 90.4% at 9% of the cost, and local Qwen3.8-27B rises from 66.8% to 92.4%. The authors note some models show no gain or worse.
Meta's 816B Muse Spark Open Scores 39; 180B Qwen3.8-Flash-Next Scores 40A user charted 11 open-weight models' Artificial Analysis scores against parameter count: Meta's Muse Spark Open (816B total, 42B active) scores 39, while Qwen3.8-Flash-Next (180B, 6B active) scores 40. The top model is 1T-parameter MiMo-V2.6-Pro at 46, followed by DeepSeek V4.1 Flash 1017 and GLM-5.3 at 45. The aggregate index says nothing about speed, long context, or vision.
Nvidia releases FP4 quant of Qwen3.8-Flash-NextA community post notes that Nvidia's FP4 quant of Qwen3.8-Flash-Next is only 102GB, smaller than the 133GB NVFP4 version, with users hoping Qwen3.8-FN-specific engines will support it.
Basalt inference engine launched with up to 2.6x speedupDevelopers released Basalt, an MIT-licensed inference engine for Blackwell GPUs, tuned for Qwen3.8 Flash-Next. It claims up to 2.6x Strata's throughput, supports dual GPUs and eight concurrent streams.
User builds 256GB VRAM rig with 8 Radeon Pro V620sA user built a roughly $2,800 inference rig with eight used Radeon Pro V620 cards totaling 256GB VRAM, running Qwen3.8-Flash-Next via a custom vLLM fork. It reached 60-100 tokens/s decode and over 3,000 tokens/s prefill, roughly 8x faster than llama.cpp.
Qwen3.8-Flash-Next gains Strix Halo support on StrataA developer released official Strix Halo support for Qwen3.8-Flash-Next on Strata, claiming best long-context decode performance, up to 1M context length, Linux-only and marked experimental.
User Shares Strata Speedup for Local LLMs but 64GB RAM ConstraintA Reddit user reported that on a dual-GPU setup (R9700, RTX 5060 Ti) with 64GB DDR5 RAM, running a quantized Qwen3.8-Flash-Next via Strata boosted inference from about 21 to 60 tokens per second, but memory usage hit 96%, preventing simultaneous ComfyUI image/video workloads, highlighting high system RAM demands of local LLMs.
Community asks if Qwen3.8 Flash Next work speeds up Qwen4A Reddit user asks whether optimization work on Qwen3.8 Flash Next will carry over to a faster Qwen4 release, citing the official Qwen blog post and hoping the effort pays off when the next model drops.
Two 96GB Ascend Cards Run Qwen3.8 Flash-NextA user built a local inference machine with two Huawei Atlas 300I Duo cards (96GB each, actually two 48GB chips per card). After vLLM, driver and operator optimizations, Qwen3.8 Flash-Next went from about 1 token/s to roughly 30 tok/s single-request and 61 tok/s at four-way concurrency, completing all 198 GPQA Diamond questions.
Open-weights releases: Victoria and Maple fine-tunesFine-tunes of Qwen3.8-Flash-Next: Victoria uses REAP to cut 44% of experts, trained in NVFP4, scores 70.0% on Terminal-Bench 2.1 and 159/164 on HumanEval, runs at 280 tok/s on one B300 with 35% fewer output tokens. Maple focuses on Canadian questions, improving official source citation from 6.0% to 62.9% and fully correct answers from 6.6% to 21.8% on 600 held-out questions.
Swift1.5-Qwen3.8-Flash-Next shines in benchmarksA user's comparison shows Swift-1.5-Qwen3.8-Flash-Next scored 41.1% first-try pass on the Aider benchmark versus 40.2% for the base model, while cutting tokens per case from 17,646 to 6,991 and time from 1,542s to 608s, sharply reducing overthinking.
Qwengram-0.8B: n-gram memory transfer cuts perplexity 5.05%A developer transferred Qwen3.8 Flash-Next's pretrained PLE n-gram memory into Qwen3.5-0.8B, keeping the backbone and ~51B-parameter memory frozen while training an R=1 reader; validation perplexity fell from 18.2759 to 17.3534, a 5.05% reduction. This is a language-model validation result, not a 5% benchmark accuracy gain.
Qwen3.8-Flash-Next Hits 65 Tokens/Second on 12GB VRAMA developer built a custom inference engine, Strata, running Qwen3.8-Flash-Next on consumer hardware including a 12GB RTX 5070, achieving about 65 tokens/s output and 430 tokens/s prompt processing, a major gain over the earlier llama.cpp setup. The engine is open-sourced, one-click installable, and currently optimized for CUDA only.
Qwen3.8-Flash-Next Hits 1M Context on Strix HaloDeveloper releases halogen 0.12.0, fixing degradation at long context depth. On a Ryzen AI Max+ 395 with 128GB RAM, decode at 1M tokens of context improved from 27.3 to 38.3 tok/s, with prefill taking about 18 minutes cold.
Offload most of Qwen3.8 KV cache to RAMA vLLM user shared on r/LocalLLaMA a method to keep most of Qwen3.8-Flash-Next's KV cache in system RAM while using a barely-fitting VRAM quant, achieving 1M context on 3×3090 GPUs with ~80 tok/s at short context, ~60 tok/s at long context, and 3,701 tok/s at a 248k prefill; patches and the model are available on their Hugging Face page.
Running Qwen3.8-Flash-Next on 12GB VRAMOn an RTX 4070 (12GB VRAM) with 64GB RAM, the author used quantization, n-gram SSD offloading, --fit settings, and MTP patches to raise Qwen3.8-Flash-Next generation from ~6 tok/s to nearly 20 tok/s, with PP around 300–350 tok/s, and documents the trade-offs and patches used.
Qwen3.8-Flash-Next supports 1M context on MLX-serveQwen3.8-Flash-Next was shown running up to ~760k context and claimed 1M-context support on MLX-serve using M5 Max with 8-bit KV cache, sustaining ~40–75 tok/s generation and requiring ~117GB peak memory.
The A.I. Daily