Inferring Relationship Using Theory of Mind in Press Diplomacy

Abstract

Diplomacy is a turn-based, non-cooperative multiplayer game. In the Press version, the relationships among players change dynamically depending on both the public situation and private communications. To ensure optimal decision-making, an agent should infer the mental states of others to identify relationships that are not explicit. In this paper, we propose the relationship theory of mind (RToM) which focuses on understanding relationships using ToM. We combine graph neural networks (GNNs) that embed the relationships and a ToM neural network (ToMnet) that discovers whom agents trust. To evaluate the RToM, we use the RToM to predict agent responses. If successful, we expect the agents can understand relationships with others to predict the acceptance of the negotiation. Our work is also applicable to other multi-agent reinforcement learning (MARL) problems featuring complex relationships, such as sequential social dilemmas (SSDs).