Auditing and Robustifying COVID-19 Misinformation Datasets via Anticontent Sampling

AAAI 2023 pp. 15260-15268

doi:10.1609/AAAI.V37I12.26780 /aaai/2023/yoo2023aaai-auditing/

Abstract

This paper makes two key contributions. First, it argues that highly specialized rare content classifiers trained on small data typically have limited exposure to the richness and topical diversity of the negative class (dubbed anticontent) as observed in the wild. As a result, these classifiers' strong performance observed on the test set may not translate into real-world settings. In the context of COVID-19 misinformation detection, we conduct an in-the-wild audit of multiple datasets and demonstrate that models trained with several prominently cited recent datasets are vulnerable to anticontent when evaluated in the wild. Second, we present a novel active learning pipeline that requires zero manual annotation and iteratively augments the training data with challenging anticontent, robustifying these classifiers.

PDF AAAI Semantic Scholar

Cite

Text

Yoo and KhudaBukhsh. "Auditing and Robustifying COVID-19 Misinformation Datasets via Anticontent Sampling." AAAI Conference on Artificial Intelligence, 2023. doi:10.1609/AAAI.V37I12.26780

Markdown

[Yoo and KhudaBukhsh. "Auditing and Robustifying COVID-19 Misinformation Datasets via Anticontent Sampling." AAAI Conference on Artificial Intelligence, 2023.](https://mlanthology.org/aaai/2023/yoo2023aaai-auditing/) doi:10.1609/AAAI.V37I12.26780

BibTeX

@inproceedings{yoo2023aaai-auditing,
  title     = {{Auditing and Robustifying COVID-19 Misinformation Datasets via Anticontent Sampling}},
  author    = {Yoo, Clay H. and KhudaBukhsh, Ashiqur R.},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2023},
  pages     = {15260-15268},
  doi       = {10.1609/AAAI.V37I12.26780},
  url       = {https://mlanthology.org/aaai/2023/yoo2023aaai-auditing/}
}