ML Anthology

Meta-Crafting: Improved Detection of Out-of-Distributed Texts via Crafting Metadata Space (Student Abstract)

Ryan Koo, Yekyung Kim, Dongyeop Kang, Jaehyung Kim
AAAI 2024 pp. 23548-23549
doi:10.1609/AAAI.V38I21.30467 /aaai/2024/koo2024aaai-meta/

Abstract

Detecting out-of-distribution (OOD) samples is crucial for robust NLP models. Recent works observe two OOD types: background shifts (style change) and semantic shifts (content change), but existing detection methods vary in effectiveness for each type. To this end, we propose Meta-Crafting, a unified OOD detection method by constructing a new discriminative feature space utilizing 7 model-driven metadata chosen empirically that well detects both types of shifts. Our experimental results demonstrate state-of-the-art robustness to both shifts and significantly improved detection on stress datasets.

PDF AAAI Semantic Scholar

Cite

Text

Koo et al. "Meta-Crafting: Improved Detection of Out-of-Distributed Texts via Crafting Metadata Space (Student Abstract)." AAAI Conference on Artificial Intelligence, 2024. doi:10.1609/AAAI.V38I21.30467

Markdown

[Koo et al. "Meta-Crafting: Improved Detection of Out-of-Distributed Texts via Crafting Metadata Space (Student Abstract)." AAAI Conference on Artificial Intelligence, 2024.](https://mlanthology.org/aaai/2024/koo2024aaai-meta/) doi:10.1609/AAAI.V38I21.30467

BibTeX

@inproceedings{koo2024aaai-meta,
  title     = {{Meta-Crafting: Improved Detection of Out-of-Distributed Texts via Crafting Metadata Space (Student Abstract)}},
  author    = {Koo, Ryan and Kim, Yekyung and Kang, Dongyeop and Kim, Jaehyung},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2024},
  pages     = {23548-23549},
  doi       = {10.1609/AAAI.V38I21.30467},
  url       = {https://mlanthology.org/aaai/2024/koo2024aaai-meta/}
}