FASE: A Fast, Accurate and Seamless Emulator for Custom Numerical Formats

Abstract

Deep Neural Networks (DNNs) have become ubiquitous in a wide range of application domains. Despite their success, training DNNs is an expensive task which has motivated the use of reduced numerical precision formats to improve performance and reduce power consumption. Emulation techniques are a good fit to understand the properties of new numerical formats on a particular workload. However, current state-of-the-art techniques are not able to perform this tasks quickly and accurately on a wide variety of workloads. We propose FASE , a Fast, Accurate and Seamless Emulator that leverages dynamic binary translation to enable emulation of arbitrary numerical formats. FASE is fast ; allowing emulation of large unmodified workloads, accurate ; emulating at instruction operand level, and seamless ; as it does not require any code modifications and works on any application or DNN framework without any language, compiler or source code access restrictions. We evaluate FASE using a wide variety of DNN frameworks and large-scale workloads. Our evaluation demonstrates that FASE achieves better accuracy than coarser-grain state-of-the-art approaches, and shows that it is able to evaluate the fidelity of multiple numerical formats and extract conclusions on their applicability.