IReEn: Reverse-Engineering of Black-Box Functions via Iterative Neural Program Synthesis

Abstract

In this work, we investigate the problem of revealing the functionality of a black-box agent. More specifically, we are interested in a formal description of the behavior of such an agent. This task is also known as reverse engineering and plays a pivotal role in software engineering, computer security, and most recently in explainability. In contrast to prior work, we do not rely on privileged information on the black box, but rather investigate the problem under a weaker assumption of having only access to inputs and outputs of the agent. We approach this problem by iteratively refining a candidate set using a generative neural program synthesis approach until we arrive at a program that mimics the agent's behavior. We assess the performance of our approach on the Karel dataset. Our results show that the proposed approach even outperforms prior work that had privileged information on the black-box function.