SortedAP: Rethinking Evaluation Metrics for Instance Segmentation
Abstract
Designing metrics for evaluating instance segmentation revolves around comprehensively considering object detection and segmentation accuracy. However, other important properties, such as sensitivity, continuity, and equality, are overlooked in the current study. In this paper, we reveal that most existing metrics have a limited resolution of segmentation quality. They are only conditionally sensitive to the change of masks or false predictions. For certain metrics, the score can change drastically in a narrow range which could provide a misleading indication of the quality gap between results. Therefore, we propose a new metric called sortedAP, which strictly decreases with both object- and pixel-level imperfections and has an uninterrupted penalization scale over the entire domain. We provide the evaluation toolkit and experiment code at https://www.github.com/looooongChen/sortedAP.
Cite
Text
Chen et al. "SortedAP: Rethinking Evaluation Metrics for Instance Segmentation." IEEE/CVF International Conference on Computer Vision Workshops, 2023. doi:10.1109/ICCVW60793.2023.00424Markdown
[Chen et al. "SortedAP: Rethinking Evaluation Metrics for Instance Segmentation." IEEE/CVF International Conference on Computer Vision Workshops, 2023.](https://mlanthology.org/iccvw/2023/chen2023iccvw-sortedap/) doi:10.1109/ICCVW60793.2023.00424BibTeX
@inproceedings{chen2023iccvw-sortedap,
title = {{SortedAP: Rethinking Evaluation Metrics for Instance Segmentation}},
author = {Chen, Long and Wu, Yuli and Stegmaier, Johannes and Merhof, Dorit},
booktitle = {IEEE/CVF International Conference on Computer Vision Workshops},
year = {2023},
pages = {3925-3931},
doi = {10.1109/ICCVW60793.2023.00424},
url = {https://mlanthology.org/iccvw/2023/chen2023iccvw-sortedap/}
}