MasakhaNEWS: News Topic Classification for African Languages
Abstract
African languages are severely under-represented in NLP research due to lack of datasets covering several NLP tasks. While there are individual language specific datasets that are being expanded to different tasks, only a handful of NLP tasks (e.g. named entity recognition and machine translation) have standardized benchmark datasets covering several geographical and typologically-diverse African languages. In this paper, we develop MasakhaNEWS --- a new benchmark dataset for news topic classification covering 16 languages widely spoken in Africa. We provide an evaluation of baseline models by training classical machine learning models and fine-tuning several language models. Furthermore, we explore several alternatives to full fine-tuning of language models that are better suited for zero-shot and few-shot learning such as cross-lingual parameter-efficient fine-tuning (like MAD-X), pattern exploiting training (PET), prompting language models (like ChatGPT), and prompt-free sentence transformer fine-tuning (SetFit and Cohere Embedding API). Our evaluation in zero-shot setting shows the potential of prompting ChatGPT for news topic classification in low-resource African languages, achieving an average performance of 70 F1 points without leveraging additional supervision like MAD-X. In few-shot setting, we show that with as little as 10 examples per label, we achieved more than 90\% (i.e. 86.0 F1 points) of the performance of full supervised training (92.6 F1 points) leveraging the PET approach.
Cite
Text
Adelani et al. "MasakhaNEWS: News Topic Classification for African Languages." ICLR 2023 Workshops: AfricaNLP, 2023.Markdown
[Adelani et al. "MasakhaNEWS: News Topic Classification for African Languages." ICLR 2023 Workshops: AfricaNLP, 2023.](https://mlanthology.org/iclrw/2023/adelani2023iclrw-masakhanews/)BibTeX
@inproceedings{adelani2023iclrw-masakhanews,
title = {{MasakhaNEWS: News Topic Classification for African Languages}},
author = {Adelani, David Ifeoluwa and Masiak, Marek and Azime, Israel Abebe and Alabi, Jesujoba Oluwadara and Tonja, Atnafu Lambebo and Mwase, Christine and Ogundepo, Odunayo and Dossou, Bonaventure F. P. and Oladipo, Akintunde and Nixdorf, Doreen and Emezue, Chris Chinenye and Al-Azzawi, sana Sabah and Sibanda, Blessing Kudzaishe and David, Davis and Ndolela, Lolwethu and Mukiibi, Jonathan and Ajayi, Tunde Oluwaseyi and Ngoli, Tatiana Moteu and Odhiambo, Brian and Owodunni, Abraham Toluwase and Obiefuna, Nnaemeka Casmir and Muhammad, Shamsuddeen Hassan and Abdullahi, Saheed Salahudeen and Yigezu, Mesay Gemeda and Gwadabe, Tajuddeen and Abdulmumin, Idris and Bame, Mahlet Taye and Awoyomi, Oluwabusayo Olufunke and Shode, Iyanuoluwa and Adelani, Tolulope Anu and Kailani, Habiba Abdulganiy and Omotayo, Abdul-Hakeem and Adeeko, Adetola and Abeeb, Afolabi and Aremu, Anuoluwapo and Samuel, Olanrewaju and Siro, Clemencia and Kimotho, Wangari and Ogbu, Onyekachi and Mbonu, Chinedu Emmanuel and Chukwuneke, Chiamaka Ijeoma and Fanijo, Samuel and Ojo, Jessica and Awosan, Oyinkansola Fiyinfoluwa and Guge, Tadesse Kebede and Sakayo, Toadoum Sari and Nyatsine, Pamela and Sidume, Freedmore and Yousuf, Oreen and Oduwole, Mardiyyah and Kimanuka, Ussen Abre and Tshinu, Kanda Patrick and Diko, Thina and Nxakama, Siyanda and Johar, Abdulmejid Tuni and Gebre, Sinodos and Mohamed, Muhidin A. and Mohamed, Shafie Abdi and Hassan, Fuad Mire and Mehamed, Moges Ahmed and Ngabire, Evrard and Stenetorp, Pontus},
booktitle = {ICLR 2023 Workshops: AfricaNLP},
year = {2023},
url = {https://mlanthology.org/iclrw/2023/adelani2023iclrw-masakhanews/}
}