INTAGS: Interactive Agent-Guided Simulation

Abstract

The development of realistic agent-based simulator (ABS) remains a challenging task, mainly due to the sequential and dynamic nature of such a multi-agent system (MAS). To fill this gap, this work proposes a metric to distinguish between real and synthetic multi-agent systems; The metric evaluation depends on the live interaction between the {\it experimental (Exp) autonomous agent} and {\it background (BG) agent(s)}, explicitly accounting for the systems' sequential and dynamic nature. Specifically, we propose to characterize the system/environment by studying the effect of a sequence of BG agents' responses to the environment state evolution, and we take such effects' differences as MAS distance metric; The effect estimation is cast as a causal inference problem since the environment evolution is confounded with the previous environment state. Importantly, we propose the \underline{Int}eractive \underline{A}gent-\underline{G}uided \underline{S}imulation (INTAGS) framework to build a realistic simulator by optimizing over this novel metric. To adapt to any environment with interactive sequential decision making agents, INTAGS formulates the simulator as a stochastic policy in reinforcement learning. Moreover, INTAGS utilizes the policy gradient update to bypass differentiating the proposed metric such that it can support non-differentiable operations of multi-agent environments. Through extensive experiments, we demonstrate the effectiveness of INTAGS on an equity stock market simulation example.