Issue 2026-09-29 · Industry · 开源 · 研究

TurboGPT trains 22KiB transformer in 13 seconds

TurboGPT trains 22KiB transformer in 13 seconds

TurboGPT, an open-source project inspired by minGPT, trains a 22KiB byte-level tiny GPT in 13 seconds using CUDA C++, under the MIT license. It requires CUDA 13.4 with a Windows-only build script, and reaches 2.5295 BPB after 1.5G training tokens on the hn1g dataset.

Hacker News12 d ago
Read original ↗