Community debate: why are Vision Transformers so popular?
A developer who deploys CNN-plus-attention models on edge devices questions the popularity of vanilla Vision Transformers, arguing that learning inductive biases from scratch seems unnecessary and that convolutional stems showed data and time efficiency gains in work such as LeViT. He suggests status quo and cheap compute may explain it, and asks whether others find vanilla ViTs excessive.