volotat

Mini-AGI: continual-learning model trained on 8GB VRAM

Developer volotat released Mini-AGI, a continual-learning model trainable on consumer hardware with 8GB of VRAM. It uses a dynamic MoE that adds and prunes experts plus batch-size-1 training on a single continuous data stream, cutting memory needs and giving individuals control over training data; the model is still processing the first of 7.8B characters, and the author reports a promising scaling curve.