OVH releases Olala mini 7A1B open model
OVH announced Olala mini, a 7B-parameter MoE model with 1B active parameters across 256 experts per layer, hybrid Differential-TPA and Mamba-3 attention and a 32K context. It runs on as little as 8GB of GPU memory at 160 tokens/s on an RTX A4500 and 50 tokens/s on CPU, and will be open-sourced on Hugging Face under Apache 2.0.