ML Anthology

Speech Is Not Enough: Interpreting Nonverbal Indicators of Common Knowledge and Engagement

Derek Palmer, Yifan Zhu, Kenneth Lai, Hannah VanderHoeven, Mariah Bradford, Ibrahim Khebour, Carlos Mabrey, Jack Fitzgerald, Nikhil Krishnaswamy, Martha Palmer, James Pustejovsky
AAAI 2025 pp. 29676-29678
doi:10.1609/AAAI.V39I28.35364 /aaai/2025/palmer2025aaai-speech/

Abstract

Our goal is to develop an AI Partner that can provide support for group problem solving and social dynamics. In multi-party working group environments, multimodal analytics is crucial for identifying non-verbal interactions of group members. In conjunction with their verbal participation, this creates an holistic understanding of collaboration and engagement that provides necessary context for the AI Partner. In this demo, we illustrate our present capabilities at detecting and tracking nonverbal behavior in student task-oriented interactions in the classroom, and the implications for tracking common ground and engagement.

PDF AAAI Semantic Scholar

Cite

Text

Palmer et al. "Speech Is Not Enough: Interpreting Nonverbal Indicators of Common Knowledge and Engagement." AAAI Conference on Artificial Intelligence, 2025. doi:10.1609/AAAI.V39I28.35364

Markdown

[Palmer et al. "Speech Is Not Enough: Interpreting Nonverbal Indicators of Common Knowledge and Engagement." AAAI Conference on Artificial Intelligence, 2025.](https://mlanthology.org/aaai/2025/palmer2025aaai-speech/) doi:10.1609/AAAI.V39I28.35364

BibTeX

@inproceedings{palmer2025aaai-speech,
  title     = {{Speech Is Not Enough: Interpreting Nonverbal Indicators of Common Knowledge and Engagement}},
  author    = {Palmer, Derek and Zhu, Yifan and Lai, Kenneth and VanderHoeven, Hannah and Bradford, Mariah and Khebour, Ibrahim and Mabrey, Carlos and Fitzgerald, Jack and Krishnaswamy, Nikhil and Palmer, Martha and Pustejovsky, James},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2025},
  pages     = {29676-29678},
  doi       = {10.1609/AAAI.V39I28.35364},
  url       = {https://mlanthology.org/aaai/2025/palmer2025aaai-speech/}
}