LiLMaps: Learnable Implicit Language Maps

Abstract

One of the current trends in robotics is to employ large language models (LLMs) to provide non-predefined commands execution and natural human-robot interaction. It is useful to have an environment map together with its language representation which can be further utilized by LLMs. Such a comprehensive scene representation enables numerous ways of interaction with the map for autonomously operating robots. In this work we present an approach that enhances incremental implicit mapping through the integration of visual-language features. Specifically we (i) propose a decoder optimization technique for implicit language maps which can be used when new objects appear on the scene and (ii) address the problem of inconsistent visual-language predictions between different viewing positions. Our experiments demonstrate the effectiveness of LiLMaps and solid improvements in performance.