Multi-Label Attention mAP Assisted Deep Feature Learning for Medical Image Classification

Abstract

Effective deep feature learning and representation is essential in training deep learning models for medical image analysis. Recent developments proposing attention mechanisms have shown promising results in improving model performance. However, their mechanism is difficult to interpret and requires architectural modifications, further reducing benefits for transfer learning and using pre-trained models. We propose an interpretability-guided attention mechanism, formulated as inductive bias operating on the loss function of the trained model. It encourages the learned deep features to yield more distinctive attention maps for multilabel classification problems. Experimental results for medical image classification show our proposed approach outperforms conventional methods utilizing different attention mechanisms while yielding class attention maps in higher agreement with clinical experts. We show how information from unlabeled images can be used directly without any model modification. The proposed approach is modular, applicable to existing network architectures used for medical imaging applications, and yields improved model performance and class attention maps.