MedMod: Multimodal Benchmark for Medical Prediction Tasks with Electronic Health Records and Chest X-Ray Scans
Abstract
Multimodal machine learning provides a myriad of opportunities for developing models that integrate multiple modalities and mimic decision-making in the real-world, such as in medical settings. However, benchmarks involving multimodal medical data are scarce, especially routinely collected modalities such as Electronic Health Records (EHR) and Chest X-ray images (CXR). To contribute towards advancing multimodal learning in tackling real-world prediction tasks, we present MedMod, a multimodal medical benchmark with EHR and CXR using publicly available datasets MIMIC-IV and MIMIC-CXR, respectively. MedMod comprises five clinical prediction tasks: clinical conditions, in-hospital mortality, decompensation, length of stay, and radiological findings. We extensively evaluate several multimodal supervised learning models and self-supervised learning frameworks, making all of our code and models open-source.
Cite
Text
Elsharief et al. "MedMod: Multimodal Benchmark for Medical Prediction Tasks with Electronic Health Records and Chest X-Ray Scans." Proceedings of the sixth Conference on Health, Inference, and Learning, 2025.Markdown
[Elsharief et al. "MedMod: Multimodal Benchmark for Medical Prediction Tasks with Electronic Health Records and Chest X-Ray Scans." Proceedings of the sixth Conference on Health, Inference, and Learning, 2025.](https://mlanthology.org/chil/2025/elsharief2025chil-medmod/)BibTeX
@inproceedings{elsharief2025chil-medmod,
title = {{MedMod: Multimodal Benchmark for Medical Prediction Tasks with Electronic Health Records and Chest X-Ray Scans}},
author = {Elsharief, Shaza and Shurrab, Saeed and Al Jorf, Baraa and Lopez, Leopoldo Julian Lechuga and Geras, Krzysztof J. and Shamout, Farah E.},
booktitle = {Proceedings of the sixth Conference on Health, Inference, and Learning},
year = {2025},
pages = {781-803},
volume = {287},
url = {https://mlanthology.org/chil/2025/elsharief2025chil-medmod/}
}