Understanding Annotator Perception: Modeling Psychological Inference from First- and Third-Person Annotations (Student Abstract)

AAAI 2025 pp. 29548-29550

doi:10.1609/AAAI.V39I28.35321 /aaai/2025/yeo2025aaai-understanding/

Abstract

Large language models (LLMs) are trained on vast amounts of publicly available text. However, the current training frameworks take for granted that these annotations are accurate reflections of the authors’ true intents. This study questions that assumption by examining the gaps between writers’ actual psychological states and the inferences made by third-party annotators. We explore how readers interpret psychological cues in text and demonstrate that third-person annotations often fail to align with first-person realities. By integrating both first- and third-person annotations, we develop computational models that reveal significant biases in how psychological states are perceived and the downstream effects these perceptions have on reader behavior. Our findings challenge the foundational assumptions of LLM training, suggesting that the reliance on potentially flawed third-person annotations could impact model accuracy and real-world applications.

AAAI Semantic Scholar

Cite

Text

Yeo and Jaidka. "Understanding Annotator Perception: Modeling Psychological Inference from First- and Third-Person Annotations (Student Abstract)." AAAI Conference on Artificial Intelligence, 2025. doi:10.1609/AAAI.V39I28.35321

Markdown

[Yeo and Jaidka. "Understanding Annotator Perception: Modeling Psychological Inference from First- and Third-Person Annotations (Student Abstract)." AAAI Conference on Artificial Intelligence, 2025.](https://mlanthology.org/aaai/2025/yeo2025aaai-understanding/) doi:10.1609/AAAI.V39I28.35321

BibTeX

@inproceedings{yeo2025aaai-understanding,
  title     = {{Understanding Annotator Perception: Modeling Psychological Inference from First- and Third-Person Annotations (Student Abstract)}},
  author    = {Yeo, Gerard Christopher and Jaidka, Kokil},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2025},
  pages     = {29548-29550},
  doi       = {10.1609/AAAI.V39I28.35321},
  url       = {https://mlanthology.org/aaai/2025/yeo2025aaai-understanding/}
}