DiffRGenNet: Difference-Aware Medical Report Generation

Abstract

Medical report generation is a critical task in healthcare, aiming to automatically pro- duce accurate diagnostic reports from medical images, thereby alleviating the burden on radiologists. However, due to the high similarity among medical images of the same anatom- ical region and the substantial variations captured from the same region across different time points for individual patients, capturing these differences poses a significant challenge. We propose a Difference-aware Report Generation Network (DiffRGenNet), which retrieves similar reports through image search, identifies differences using the Feature Diff module, and dynamically orchestrates global and local dependencies via the FlexiRoute Aggregation Module to determine the optimal routing path for each sample, selecting the most suitable report to describe the variations and connections. Finally, by leveraging the consistency of classification information and the discrepancy information from the diff module, DiffR- GenNet enhances the ability to learn differences in rare diseases, generating more precise reports. Experiments demonstrate that DiffRGenNet outperforms existing methods on the MIMIC-CXR and IU X-Ray datasets, confirming its effectiveness and potential.