3.87B MoE trained from scratch on 86.5B tokens

A developer trained Apex-2, a 3.87B-parameter MoE (1.45B active) from scratch on just 86.5B pretrain tokens, reaching HumanEval 43.9 but MMLU 28.6; DPO hurt scores and was dropped, and the model is open-sourced.

• Pulse Local Launches On-Device AI Budgeting App for Mac2026-10-14
• Qwen-family LLMs becoming backbone of audio models2026-09-30
• Block removal for LLM pruning framed as Ising optimization2026-09-21
• User builds home server from old PC and mining GPUs, runs Qwen3 27B at ~30 tokens/s2026-09-19