Partial Channel Dependence with Channel Masks for Time Series Foundation Model

Abstract

While advances in foundation models have extended to the time series domain, they have primarily focused on designing model architectures to address external heterogeneity between datasets, e.g., varying numbers of channels, often overlooking internal heterogeneity, e.g., varying channel dependencies. In this work, we introduce the concept of partial channel dependence (PCD), which enables a more sophisticated adjustment of channel dependencies based on dataset-specific information. To achieve PCD, we propose a channel mask that captures the relationships between channels within a dataset using two key components: 1) a correlation matrix that encodes relative dependencies between channels, and 2) domain parameters that learn the absolute dependencies specific to each dataset, refining the correlation matrix. We validate the effectiveness of our method across various tasks, including forecasting, classification, imputation, and anomaly detection.