LITMUS Predictor: An AI Assistant for Building Reliable, High-Performing and Fair Multilingual NLP Systems
Abstract
Pre-trained multilingual language models are gaining popularity due to their cross-lingual zero-shot transfer ability, but these models do not perform equally well in all languages. Evaluating task-specific performance of a model in a large number of languages is often a challenge due to lack of labeled data, as is targeting improvements in low performing languages through few-shot learning. We present a tool - LITMUS Predictor - that can make reliable performance projections for a fine-tuned task-specific model in a set of languages without test and training data, and help strategize data labeling efforts to optimize performance and fairness objectives.
Cite
Text
Srinivasan et al. "LITMUS Predictor: An AI Assistant for Building Reliable, High-Performing and Fair Multilingual NLP Systems." AAAI Conference on Artificial Intelligence, 2022. doi:10.1609/AAAI.V36I11.21736Markdown
[Srinivasan et al. "LITMUS Predictor: An AI Assistant for Building Reliable, High-Performing and Fair Multilingual NLP Systems." AAAI Conference on Artificial Intelligence, 2022.](https://mlanthology.org/aaai/2022/srinivasan2022aaai-litmus/) doi:10.1609/AAAI.V36I11.21736BibTeX
@inproceedings{srinivasan2022aaai-litmus,
title = {{LITMUS Predictor: An AI Assistant for Building Reliable, High-Performing and Fair Multilingual NLP Systems}},
author = {Srinivasan, Anirudh and Kholkar, Gauri and Kejriwal, Rahul and Ganu, Tanuja and Dandapat, Sandipan and Sitaram, Sunayana and Santhanam, Balakrishnan and Aditya, Somak and Bali, Kalika and Choudhury, Monojit},
booktitle = {AAAI Conference on Artificial Intelligence},
year = {2022},
pages = {13227-13229},
doi = {10.1609/AAAI.V36I11.21736},
url = {https://mlanthology.org/aaai/2022/srinivasan2022aaai-litmus/}
}