New Tools for Exploring the Souls of LLMs
The piece notes mechanistic interpretability has grown from a handful of researchers in 2021 to hundreds of active contributors, while Anthropic and OpenAI face criticism for spending on models rather than safety research. The author built a tool, TinyGPT Visualizer, to help outsiders inspect models.