ModernTCN Revisited: A Critical Look at the Experimental Setup in General Time Series Analysis

Abstract

While numerous time series models claim state-of-the-art performance, their evaluation often relies on flawed experimental setups, leading to questionable conclusions. This study provides a critical re-evaluation of this landscape, using ModernTCN as a case study. We conduct a rigorous and extended benchmark, correcting methodological issues related to data loading, validation, and evaluation methods, and show that performance claims are sensitive to these details. Additionally, we find that ModernTCN overlooks a line of research in global convolutional models, and our comparison reveals that despite claims of an enlarged effective receptive field (ERF), it falls short of these methods. More than a critique, we introduce an architectural innovation: by embedding irregularly sampled data with a continuous kernel convolution and processing it with the ModernTCN backbone, we achieve new state-of-the-art performance on the challenging PhysioNet 2019 dataset. This work not only provides a robust reassessment of ModernTCN but also serves as an audit of the commonly used general time series analysis experimental setup, which includes tasks such as forecasting, imputation, classification, and anomaly detection.