Sebastian Raschka is a machine learning researcher and author known for from-scratch tutorials such as Build a Large Language Model (from Scratch). Recent coverage centers on his teaching and commentary: he released an eight-video, 11-plus-hour reasoning-from-scratch course that builds reasoning models from a pre-trained LLM in Python and PyTorch, covering evaluation, inference-time scaling, reinforcement learning and distillation. He also published a visual guide tracing text classification from bag-of-words to Transformers, Jev and calibration, and commented on MiMo-V2.6's grouped query attention design, Muon versus AdamW, and the OpenArch PyTorch repository.
Raschka releases from-scratch reasoning model video courseSebastian Raschka released an eight-video, 11-plus-hour course that builds reasoning models from a pre-trained LLM in Python and PyTorch, covering evaluation, inference-time scaling, reinforcement learning and distillation. It is a sequel to his earlier coding LLMs course.
The complete reasoning-from-scratch video series is now up (and linked) here: (Taking a short break but will be back with LLM model and research updates in November!)
Raschka Traces Evolution of Text Classification ModelsSebastian Raschka published a visual guide tracing text classification from bag-of-words through RNNs, CNNs, Transformers and Jev-like APIs to calibration, arguing the recently released Jev is essentially a classifier that is faster and cheaper than general LLMs yet more general than task-specific models; the article is not an endorsement.
The main appeal of open-source agent harnesses isn't that they are free, but that we can inspect what they are doing on our computers.
MiMo-V2.6 is "simply" the best (for now). Despite its simple architecture design it's currently No.1 in the open-weight benchmarks (weighted average). With "simple," I mean a classic Grouped Query Attention (GQA) with Sliding Window Attention (SWA) at a tiny…
Interesting new insights into the old AdamW vs Muon debate: Muon seems to do better because it reduces memorization. [Quoting @CalcCon]: 𝗔𝗱𝗮𝗺𝗪 𝘃𝘀. 𝗠𝘂𝗼𝗻: α, 𝗢𝘃𝗲𝗿𝗳𝗶𝘁𝘁𝗶𝗻𝗴, 𝗮𝗻𝗱 𝗠𝗲𝗺𝗼𝗿𝗶𝘇𝗮𝘁𝗶𝗼𝗻. A quick update on our WeightWatcher…
OpenArch — PyTorch implementations of modern LLM architecturesOpenArch is a hand-written PyTorch repository implementing modern open-source LLM architectures one model at a time, prioritizing readability and learning. It documents various attention types, normalization choices, and positional encodings; some implementations support forward passes.
Big overhaul on DeepSeek V4.1 using an encoder-decoder setup. Tbh they should have called it DeepSeek V5! Super cool and refreshing, though! [Quoting @deepseek_ai]: 🚀 Introducing DeepSeek-V4.1-Flash: smarter, faster, more efficient. 🔹 Introducing the smalle…
Nice showcase that interesting LLM work can be done on single GPU! [Quoting @gpjt]: I extended the GPT-2-style code from @rasbt's "Build a Large Language Model (from Scratch)" so that it was a 6-expert (2 active) mixture-of-experts, and trained it from scratc…
The A.I. Daily