Inductive Biases for Zero-Shot Systematic Generalization in Language-Informed Reinforcement Learning

Abstract

Sample efficiency and systematic generalization remain persistent challenges in reinforcement learning. While previous studies demonstrate that incorporating natural language with other observation modalities enhances generalization and sample efficiency due to its compositional and open-ended nature, effectively leveraging these properties requires robust language grounding mechanisms. To address this, we propose InstructionConditionedMOdular network (ICMO) by introducing language entrance and memory feedback techniques on top of an existing modular and sparse architecture, NPS. The memory feedback mechanism aggregates high-level information, guides selective attention in NPS via attentional feedback, and strengthens the decision-making process in the presence of language guidance. ICMO achieves superior performance compared to previous methods during our rigorous experiments, demonstrating near-zero generalization gap that highlights its robustness. Additionally, an extensive ablation study confirms the contributions of these techniques to improving generalization, sample efficiency, and training stability.