Meta trains byte-level small models, predicted to beat token models

Meta trained ~1B-parameter byte-level models distilled from Llama 3 8B; they lag token models early but keep improving. Extrapolated curves suggest they could beat Llama 3.2 1B and Gemma 3 1B with about 6x less text, though real runs still favored tokens.

• AI Debt Monster: Hidden Risks in Infrastructure Spending2026-10-24
• Musk wants SpaceX-Tesla merger; Meta deal to keep Zuckerberg2026-10-23
• Meta delays new business terms after publisher pushback on AI scraping2026-10-23
• Meta's 816B Muse Spark Open Scores 39; 180B Qwen3.8-Flash-Next Scores 402026-10-23
• Firms tighten AI budgets, pressuring OpenAI and Anthropic2026-10-23