Photogrammetry and VR for Comparing 2D and Immersive Linguistic Data Collection (Student Abstract)

Jacob Rubinstein, Cynthia Matuszek, Don Engel

AAAI 2023 pp. 16312-16313

doi:10.1609/AAAI.V37I13.27016 /aaai/2023/rubinstein2023aaai-photogrammetry/

Abstract

The overarching goal of this work is to enable the collection of language describing a wide variety of objects viewed in virtual reality. We aim to create full 3D models from a small number of ‘keyframe’ images of objects found in the publicly available Grounded Language Dataset (GoLD) using photogrammetry. We will then collect linguistic descriptions by placing our models in virtual reality and having volunteers describe them. To evaluate the impact of virtual reality immersion on linguistic descriptions of the objects, we intend to apply contrastive learning to perform grounded language learning, then compare the descriptions collected from images (in GoLD) versus our models.

PDF AAAI Semantic Scholar

Cite

Text

Rubinstein et al. "Photogrammetry and VR for Comparing 2D and Immersive Linguistic Data Collection (Student Abstract)." AAAI Conference on Artificial Intelligence, 2023. doi:10.1609/AAAI.V37I13.27016

Markdown

[Rubinstein et al. "Photogrammetry and VR for Comparing 2D and Immersive Linguistic Data Collection (Student Abstract)." AAAI Conference on Artificial Intelligence, 2023.](https://mlanthology.org/aaai/2023/rubinstein2023aaai-photogrammetry/) doi:10.1609/AAAI.V37I13.27016

BibTeX

@inproceedings{rubinstein2023aaai-photogrammetry,
  title     = {{Photogrammetry and VR for Comparing 2D and Immersive Linguistic Data Collection (Student Abstract)}},
  author    = {Rubinstein, Jacob and Matuszek, Cynthia and Engel, Don},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2023},
  pages     = {16312-16313},
  doi       = {10.1609/AAAI.V37I13.27016},
  url       = {https://mlanthology.org/aaai/2023/rubinstein2023aaai-photogrammetry/}
}