The A.I. Daily
FlashAttention kernel auto-derived from ~25 lines of SQL
A demo shows a FlashAttention kernel being automatically derived from roughly 25 lines of SQL expressing vanilla attention. It relies on three SQL extensions: xarray-sql, the JAX-style autodiff extension ddx, and the einfold query planner; the author argues relational algebra offers a better way to program accelerators.