ML Anthology

Learning Geometric Representations of Objects via Interaction

Alfredo Reichlin, Giovanni Luca Marchetti, Hang Yin, Anastasiia Varava, Danica Kragic
ECML-PKDD 2023 pp. 629-644
doi:10.1007/978-3-031-43421-1_37 /ecmlpkdd/2023/reichlin2023ecmlpkdd-learning/

Abstract

We address the problem of learning representations from observations of a scene involving an agent and an external object the agent interacts with. To this end, we propose a representation learning framework extracting the location in physical space of both the agent and the object from unstructured observations of arbitrary nature. Our framework relies on the actions performed by the agent as the only source of supervision, while assuming that the object is displaced by the agent via unknown dynamics. We provide a theoretical foundation and formally prove that an ideal learner is guaranteed to infer an isometric representation, disentangling the agent from the object and correctly extracting their locations. We evaluate empirically our framework on a variety of scenarios, showing that it outperforms vision-based approaches such as a state-of-the-art keypoint extractor. We moreover demonstrate how the extracted representations enable the agent to solve downstream tasks via reinforcement learning in an efficient manner.

PDF ECML-PKDD Semantic Scholar

Cite

Text

Reichlin et al. "Learning Geometric Representations of Objects via Interaction." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2023. doi:10.1007/978-3-031-43421-1_37

Markdown

[Reichlin et al. "Learning Geometric Representations of Objects via Interaction." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2023.](https://mlanthology.org/ecmlpkdd/2023/reichlin2023ecmlpkdd-learning/) doi:10.1007/978-3-031-43421-1_37

BibTeX

@inproceedings{reichlin2023ecmlpkdd-learning,
  title     = {{Learning Geometric Representations of Objects via Interaction}},
  author    = {Reichlin, Alfredo and Marchetti, Giovanni Luca and Yin, Hang and Varava, Anastasiia and Kragic, Danica},
  booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
  year      = {2023},
  pages     = {629-644},
  doi       = {10.1007/978-3-031-43421-1_37},
  url       = {https://mlanthology.org/ecmlpkdd/2023/reichlin2023ecmlpkdd-learning/}
}