To Adapt or Not to Adapt? Real-Time Adaptation for Semantic Segmentation
Abstract
The goal of Online Domain Adaptation for semantic segmentation is to handle unforeseeable domain changes that occur during deployment, like sudden weather events. However, the high computational costs associated with brute-force adaptation make this paradigm unfeasible for real-world applications. In this paper we propose HAMLET, a Hardware-Aware Modular Least Expensive Training framework for real-time domain adaptation. Our approach includes a hardware-aware back-propagation orchestration agent (HAMT) and a dedicated domain-shift detector that enables active control over when and how the model is adapted (LT). Thanks to these advancements, our approach is capable of performing semantic segmentation while simultaneously adapting at more than 29FPS on a single consumer-grade GPU. Our framework's encouraging accuracy and speed trade-off is demonstrated on OnDA and SHIFT benchmarks through experimental results.
Cite
Text
Colomer et al. "To Adapt or Not to Adapt? Real-Time Adaptation for Semantic Segmentation." International Conference on Computer Vision, 2023. doi:10.1109/ICCV51070.2023.01517Markdown
[Colomer et al. "To Adapt or Not to Adapt? Real-Time Adaptation for Semantic Segmentation." International Conference on Computer Vision, 2023.](https://mlanthology.org/iccv/2023/colomer2023iccv-adapt/) doi:10.1109/ICCV51070.2023.01517BibTeX
@inproceedings{colomer2023iccv-adapt,
title = {{To Adapt or Not to Adapt? Real-Time Adaptation for Semantic Segmentation}},
author = {Colomer, Marc Botet and Dovesi, Pier Luigi and Panagiotakopoulos, Theodoros and Carvalho, Joao Frederico and Härenstam-Nielsen, Linus and Azizpour, Hossein and Kjellström, Hedvig and Cremers, Daniel and Poggi, Matteo},
booktitle = {International Conference on Computer Vision},
year = {2023},
pages = {16548-16559},
doi = {10.1109/ICCV51070.2023.01517},
url = {https://mlanthology.org/iccv/2023/colomer2023iccv-adapt/}
}