Sebastian Raschka is a machine learning practitioner and educator known for tutorials and open-source training code such as “Build a Large Language Model (from Scratch)”. Recent coverage mentioned the OpenArch hand-written PyTorch implementations of modern LLM architectures, DeepSeek V4.1’s encoder-decoder update, and a single-GPU LLM showcase that extended rasbt’s GPT-2–style code into a 6-expert (2 active) mixture-of-experts trained from scratch.
OpenArch — PyTorch implementations of modern LLM architecturesOpenArch is a hand-written PyTorch repository implementing modern open-source LLM architectures one model at a time, prioritizing readability and learning. It documents various attention types, normalization choices, and positional encodings; some implementations support forward passes.
Big overhaul on DeepSeek V4.1 using an encoder-decoder setup. Tbh they should have called it DeepSeek V5! Super cool and refreshing, though! https://t.co/r00s3Cl29m [Quoting @deepseek_ai]: 🚀 Introducing DeepSeek-V4.1-Flash: smarter, faster, more efficient. …
Nice showcase that interesting LLM work can be done on single GPU! [Quoting @gpjt]: I extended the GPT-2-style code from @rasbt's "Build a Large Language Model (from Scratch)" so that it was a 6-expert (2 active) mixture-of-experts, and trained it from scratc…
The A.I. Daily