Sebastian Raschka @rasbt

Interesting new insights into the old AdamW vs Muon debate: Muon seems to do better because it reduces memorization. [引用 @CalcCon]: 𝗔𝗱𝗮𝗺𝗪 𝘃𝘀. 𝗠𝘂𝗼𝗻: α, 𝗢𝘃𝗲𝗿𝗳𝗶𝘁𝘁𝗶𝗻𝗴, 𝗮𝗻𝗱 𝗠𝗲𝗺𝗼𝗿𝗶𝘇𝗮𝘁𝗶𝗼𝗻. A quick update on our WeightWatcher experiments comparing AdamW with the newer Muon optimizer. We trained a single-head nanoGPT model with both optimizers across five seeds. We then used WeightWatcher to study the weight-matrix spectra and several forms of memorization. The top plot shows the average raw α.…

• Raschka releases from-scratch reasoning model video course2026-10-26
• Raschka Traces Evolution of Text Classification Models2026-09-30
• OpenArch — PyTorch implementations of modern LLM architectures2026-09-14