Multivariate Time Series Anomaly Prediction Based on Forecasting and Reconstruction Using Transformer with Temporal and Feature-Wise Attention

Abstract

Anomaly detection has been actively studied, enabling the high-accuracy detection of anomalies. However, because anomaly detection assumes that an anomaly has already occurred, detecting future anomalies before they occur and preventing them from happening is impossible. Therefore, we develop a Transformer-based Anomaly Prediction (TranAP) method, which is designed to detect future anomalies. TranAP predicts future values from previous time series and uses reconstruction techniques to detect signs of anomalies using the predicted results. Detecting these precursors requires a correct understanding of the temporal characteristics of the multivariate time series (MTS). Because the timing of behavior leading to an anomaly may differ for each feature, we apply multi-head attention (ATTN) in the time dimension for each feature. Additionally, TranAP captures the dependencies between different features that the conventional ATTN could not. Because the effect of ATTN is partially diminished within the attention block, even after improvement to capture detailed information in MTS, we modify the operation of the block to preserve this effect. We demonstrate the effectiveness of TranAP by comparing it with state-of-the-art models. This improved attention mechanism of TranAP allows for a better understanding of behavior that leads to anomalies.