Near-Optimal Interdiction of Factored MDPs

Abstract

Stackelberg games have been widely used to model interactions between attackers and defenders in a broad array of security domains. One related approach involves plan interdiction, whereby a defender chooses a subset of actions to block (remove), and the attacker constructs an optimal plan in response. In previous work, this approach has been introduced in the context of Markov decision processes (MDPs). The key challenge, however, is that the state space of MDPs grows exponentially in the number of state variables. We propose a novel scalable MDP interdiction framework which makes use of factored representation of state, using Fourier representation for representing a value function over a boolean space. We demonstrate that our approach is significantly more scalable, while resulting in near-optimal interdiction decisions.