The A.I. Daily
300B-Class MoE Models Run on a Single 128GB Mini PC
Developers released Kyojin, an ExLlamaV3-based engine for AMD Strix Halo (ROCm), letting two 300B-class MoE models each run on one 128GB mini PC. GLM-5.3-Flash prefills at ~580 tok/s; MiMo-V2.6-Flash decodes up to 44 tok/s, with KLD and Top-1 agreement versus official FP8 reported.