A Human-Centric Assessment of the Usefulness of Attribution Methods in Computer Vision

Abstract

Explainable AI has emerged to assure the validity of predictions by explaining produced decisions; nevertheless, it arrived with its own challenges of objectively evaluating explanations or assessing their usefulness to end users. Human-centric evaluation methods, such as simulatability, that are based on human subject studies have been shown to produce contradictory findings on the usefulness of explanations. It remains unclear if these contradictory results are caused by uncontrolled confounders: external factors that influence the measured quantity. To enable a reliable, human-centric, and trustworthy evaluation, we propose a generic assessment framework that allows researchers to measure the usefulness of XAI methods, such as attribution-based explanations, in an experimental setting with a reduced set of confounders. Applied simultaneously to multiple XAI techniques, the framework returns a usefulness ranking of the XAI models and also compares them with a human baseline. In a large-scale subject study, our results show that the acceptance rate increases from 64.2% without explanations to 86% with XAI methods and 92.7% when given human explanations. One particular model obtains indistinguishable results from human explanations, raising the acceptance score to 94.5%.