OpenArch — PyTorch implementations of modern LLM architectures

OpenArch is a hand-written PyTorch repository implementing modern open-source LLM architectures one model at a time, prioritizing readability and learning. It documents various attention types, normalization choices, and positional encodings; some implementations support forward passes.

• Foundation Model Engineering textbook2026-09-14
• Advice for hobbyists: don’t let FOMO drive local LLM spending2026-09-09