Differentiable Weighted Automata

Abstract

Weighted automata have been used extensively for studying quantitative properties of systems, modeling the behavior of probabilistic systems, and various text, speech, and image processing domains. Moreover, they have become popular in reinforcement learning for task specifications in the form of reward machines. This is due to their ability to assign a quantitative score to some sequence of inputs in the problem domain. While the weighted automaton formalism is powerful and well-studied in literature, its inherently discrete structure makes it difficult to use in gradient-based pipelines. In this paper, we present a systematic framework for designing differentiable weighted automata that can leverage automatic differentiation tools to compute the gradient of the weight calculated by a weighted automaton with respect to its input sequence.