HPExplorer: XAI Method to Explore the Relationship Between Hyperparameters and Model Performance

Abstract

Hyperparameters (HPs) play a central role in the performance of machine learning (ML) models, governing model structure, regularization, and convergence properties. Understanding the intricate relationship between HP configurations and model performance is essential for ML practitioners, especially those with limited expertise, to develop effective models that produce satisfactory results. This paper introduces HyperParameter Explorer (HPExplorer), a semi-automated eXplainable AI (XAI) method, to support ML practitioners to explore this relationship. HPExplorer integrates an automated HP discovery algorithm with an interactive visual exploration component. The HP discovery algorithm identifies performance-consistent subspaces within the HP space, where models perform similarly despite minor variations in HP configurations. The interactive visual exploration component enables users to explore the discovered performance-consistent subspaces using an interactive 2-D projection called Star Coordinate. Users can also compare HP configurations from different subspaces to explore their impact on model performance. We developed HPExplorer in close collaboration with ML practitioners, particularly geoscientists, using ML in their research. Initial feedback from scientists using HPExplorer in real-world scenarios indicates that HPExploer enhances the transparency in configuring HPs and increases the confidence of users in their decisions.