PLUTO: Pathology-Universal Transformer
Abstract
Pathology images provide a unique challenge for computer-vision-based analysis: a single whole slide image is gigapixel-sized and often contains hundreds of thousands to millions of objects of interest across multiple resolutions. In this work, we propose PathoLogy Universal TransfOrmer (PLUTO): a light-weight pathology foundation model (FM) that is pre-trained on a diverse dataset of 195 million image tiles collected from multiple sites. We design task-specific adaptation heads that utilize PLUTO's output embeddings for tasks ranging from subcellular- to slide-scale, including instance segmentation, tile classification, and slide-level prediction. We find that PLUTO matches or outperforms existing task-specific baselines and pathology-specific FMs, some of which use orders-of-magnitude larger datasets and model sizes. Our findings present a path towards a universal embedding to power pathology image analysis, and motivate further exploration around pathology FMs in terms of data diversity, architectural improvements, sample efficiency, and practical deployability in real-world applications.
Cite
Text
Juyal et al. "PLUTO: Pathology-Universal Transformer." ICML 2024 Workshops: AccMLBio, 2024.Markdown
[Juyal et al. "PLUTO: Pathology-Universal Transformer." ICML 2024 Workshops: AccMLBio, 2024.](https://mlanthology.org/icmlw/2024/juyal2024icmlw-pluto/)BibTeX
@inproceedings{juyal2024icmlw-pluto,
title = {{PLUTO: Pathology-Universal Transformer}},
author = {Juyal, Dinkar and Padigela, Harshith and Shah, Chintan and Shenker, Daniel and Harguindeguy, Natalia and Liu, Yi and Martin, Blake and Zhang, Yibo and Nercessian, Michael and Markey, Miles and Finberg, Isaac and Luu, Kelsey and Borders, Daniel and Javed, Syed Ashar and Krause, Emma L and Biju, Raymond and Sood, Aashish and Ma, Allen and Nyman, Jackson and Shamshoian, John and Chhor, Guillaume and Sanghavi, Darpan and Thibault, Marc and Yu, Limin and Najdawi, Fedaa and Hipp, Jennifer A. and Fahy, Darren and Glass, Benjamin and Walk, Eric and Abel, John and Pokkalla, Harsha Vardhan and Beck, Andrew H. and Grullon, Sean},
booktitle = {ICML 2024 Workshops: AccMLBio},
year = {2024},
url = {https://mlanthology.org/icmlw/2024/juyal2024icmlw-pluto/}
}