Using Rashomon Sets for Robust Active Learning

Abstract

Active learning is based on selecting informative data points to enhance model predictions, often using uncertainty as a selection criterion. However, when ensemble models such as random forests are used, there is a risk of the ensemble containing models with poor predictive accuracy or duplicates with the same interpretation. To address this, we develop a novel approach to only ensemble only the set of near-optimal models called the Rashomon set in order to guide the active learning process. We demonstrate how taking a Rashomon approach can improve not only the accuracy and rate of convergence of the active learning procedure, but can also lead to improved interpretability compared to traditional approaches.