Every MMLU story collected by The AI Daily, 2 so far, newest first, refreshed hourly.
3.87B MoE trained from scratch on 86.5B tokensA developer trained Apex-2, a 3.87B-parameter MoE (1.45B active) from scratch on just 86.5B pretrain tokens, reaching HumanEval 43.9 but MMLU 28.6; DPO hurt scores and was dropped, and the model is open-sourced.
Block removal for LLM pruning framed as Ising optimizationResearchers reformulate transformer block removal as constrained binary optimization mapped to an Ising spin-glass model, letting them rank pruning candidates without benchmarking each one. At 50% compression of Llama-3.3-70B-Instruct, it gains nearly 23 percentage points on MMLU over the best competing method.
The A.I. Daily