OpenBMB runs 35B MoE model locally on iPhone
OpenBMB detailed an engineering approach that runs a 35B-A3B MoE model at 20.51 tok/s on an iPhone 18 Pro Max, 6.07x faster than baseline. Its Lookahead Expert Scheduling preloads predicted expert weights to hide storage bottlenecks without skipping experts or changing routing, using about 3 GB peak memory; the code is open source.