The Casual Conversations V2 Dataset : A Diverse, Large Benchmark for Measuring Fairness and Robustness in Audio/vision/speech Models

Bilal Porgali, Vítor Albiero, Jordan Ryda, Cristian Canton-Ferrer, Caner Hazirbas

CVPRW 2023 pp. 10-17

doi:10.1109/CVPRW59228.2023.00006 /cvprw/2023/porgali2023cvprw-casual/

Abstract

This paper introduces a new large consent-driven dataset aimed at assisting in the evaluation of algorithmic bias and robustness of computer vision and audio speech models in regards to 11 attributes that are self-provided or labeled by trained annotators. The dataset includes 26,467 videos of 5,567 unique paid participants, with an average of almost 5 videos per person, recorded in Brazil, India, Indonesia, Mexico, Vietnam, Philippines, and the USA, representing diverse demographic characteristics. The participants agreed for their data to be used in assessing fairness of AI models and provided self-reported age, gender, language/dialect, disability status, physical adornments, physical attributes and geo-location information, while trained annotators labeled apparent skin tone using the Fitzpatrick Skin Type and Monk Skin Tone scales, and voice timbre. Annotators also labeled for different recording setups and per-second activity annotations.

CVPRW Semantic Scholar

Cite

Text

Porgali et al. "The Casual Conversations V2 Dataset : A Diverse, Large Benchmark for Measuring Fairness and Robustness in Audio/vision/speech Models." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2023. doi:10.1109/CVPRW59228.2023.00006

Markdown

[Porgali et al. "The Casual Conversations V2 Dataset : A Diverse, Large Benchmark for Measuring Fairness and Robustness in Audio/vision/speech Models." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2023.](https://mlanthology.org/cvprw/2023/porgali2023cvprw-casual/) doi:10.1109/CVPRW59228.2023.00006

BibTeX

@inproceedings{porgali2023cvprw-casual,
  title     = {{The Casual Conversations V2 Dataset : A Diverse, Large Benchmark for Measuring Fairness and Robustness in Audio/vision/speech Models}},
  author    = {Porgali, Bilal and Albiero, Vítor and Ryda, Jordan and Canton-Ferrer, Cristian and Hazirbas, Caner},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
  year      = {2023},
  pages     = {10-17},
  doi       = {10.1109/CVPRW59228.2023.00006},
  url       = {https://mlanthology.org/cvprw/2023/porgali2023cvprw-casual/}
}