Transolver is a Linear Transformer: Revisiting Physics-Attention through the Lens of Linear Attention
My rating
Key insight: slice-and-deslice operations, not interactions between slices, drive Transolver's effectiveness. The slice attention may even hurt performance.
Contribution: A two-step transformation redesigns Physics-Attention into canonical linear attention, delivering Pareto improvements (better accuracy at lower compute) on every benchmark tested.
Tags: