BigBio: A Framework for Data-Centric Biomedical Natural Language Processing
Abstract
Training and evaluating language models increasingly requires the construction of meta-datasets -- diverse collections of curated data with clear provenance. Natural language prompting has recently lead to improved zero-shot generalization by transforming existing, supervised datasets into a variety of novel instruction tuning tasks, highlighting the benefits of meta-dataset curation. While successful in general-domain text, translating these data-centric approaches to biomedical language modeling remains challenging, as labeled biomedical datasets are significantly underrepresented in popular data hubs. To address this challenge, we introduce BigBio a community library of 126+ biomedical NLP datasets, currently covering 13 task categories and 10+ languages. BigBio facilitates reproducible meta-dataset curation via programmatic access to datasets and their metadata, and is compatible with current platforms for prompt engineering and end-to-end few/zero shot language model evaluation. We discuss our process for task schema harmonization, data auditing, contribution guidelines, and outline two illustrative use cases: zero-shot evaluation of biomedical prompts and large-scale, multi-task learning. BigBio is an ongoing community effort and is available at https://github.com/bigscience-workshop/biomedical
Cite
Text
Fries et al. "BigBio: A Framework for Data-Centric Biomedical Natural Language Processing." Neural Information Processing Systems, 2022.Markdown
[Fries et al. "BigBio: A Framework for Data-Centric Biomedical Natural Language Processing." Neural Information Processing Systems, 2022.](https://mlanthology.org/neurips/2022/fries2022neurips-bigbio/)BibTeX
@inproceedings{fries2022neurips-bigbio,
title = {{BigBio: A Framework for Data-Centric Biomedical Natural Language Processing}},
author = {Fries, Jason and Weber, Leon and Seelam, Natasha and Altay, Gabriel and Datta, Debajyoti and Garda, Samuele and Kang, Sunny and Su, Rosaline and Kusa, Wojciech and Cahyawijaya, Samuel and Barth, Fabio and Ott, Simon and Samwald, Matthias and Bach, Stephen and Biderman, Stella and Sänger, Mario and Wang, Bo and Callahan, Alison and Periñán, Daniel León and Gigant, Théo and Haller, Patrick and Chim, Jenny and Posada, Jose and Giorgi, John and Sivaraman, Karthik Rangasai and Pàmies, Marc and Nezhurina, Marianna and Martin, Robert and Cullan, Michael and Freidank, Moritz and Dahlberg, Nathan and Mishra, Shubhanshu and Bose, Shamik and Broad, Nicholas and Labrak, Yanis and Deshmukh, Shlok and Kiblawi, Sid and Singh, Ayush and Vu, Minh Chien and Neeraj, Trishala and Golde, Jonas and del Moral, Albert Villanova and Beilharz, Benjamin},
booktitle = {Neural Information Processing Systems},
year = {2022},
url = {https://mlanthology.org/neurips/2022/fries2022neurips-bigbio/}
}