Steerable Transformers for Volumetric Data
Abstract
We introduce Steerable Transformers, an extension of the Vision Transformer mechanism that maintains equivariance to the special Euclidean group $\mathrm{SE}(d)$. We propose an equivariant attention mechanism that operates on features extracted by steerable convolutions. Operating in Fourier space, our network utilizes Fourier space non-linearities. Our experiments in both two and three dimensions show that adding steerable transformer layers to steerable convolutional networks enhances performance.
Cite
Text
Kundu and Kondor. "Steerable Transformers for Volumetric Data." Proceedings of the 42nd International Conference on Machine Learning, 2025.Markdown
[Kundu and Kondor. "Steerable Transformers for Volumetric Data." Proceedings of the 42nd International Conference on Machine Learning, 2025.](https://mlanthology.org/icml/2025/kundu2025icml-steerable/)BibTeX
@inproceedings{kundu2025icml-steerable,
title = {{Steerable Transformers for Volumetric Data}},
author = {Kundu, Soumyabrata and Kondor, Risi},
booktitle = {Proceedings of the 42nd International Conference on Machine Learning},
year = {2025},
pages = {32011-32031},
volume = {267},
url = {https://mlanthology.org/icml/2025/kundu2025icml-steerable/}
}