Team applies recursive self-improvement to pre-training, open-sources 801-30B-A3B
Researchers say they are first to apply recursive self-improvement (RSI) to a from-scratch frontier pre-training run, decomposing the loop into smaller closeable loops because a run costs hundreds of thousands of GPU-hours and cannot be repeated. The result, 801-30B-A3B-Base, is a 30B-A3B MoE trained on 11.8T tokens in three months. It beats Qwen3-30B-A3B and Nemotron-3.5-Lightning-30B-A3B, trailing only the larger Qwen3.5-35B-A3B; the model and SFT data are open-sourced.