Analyzing and Correcting Biased Machine Learning-Based Tuning of Weight Shrinkage in Forecast Combination

Abstract

A forecast combination typically corresponds to a weighted average of individual forecasts and aims at increasing predictive accuracy. Application fields include business, economics, information systems such as recommender systems and financial portfolios. One popular weighting approach used in various studies is to learn weights optimal on past data (optimal weights) and shrink them towards equal weights to mitigate overfitting. The required shrinkage hyperparameter is usually tuned by machine learning-based techniques like K-fold cross-validation (CV). This paper shows that CV-tuned shrinkage levels are generally biased: Depending on the characteristics (parameters) of training forecast data (e.g., number of forecasters, error correlations, spread in predictive ability, training set size, number of CV-folds), such approaches lead to systematic over- or undershrinkage. The impact of different parameters on these biases is studied on large sets of synthetically generated data and a model is trained to predict the bias (direction and degree) by using data characteristics as features. This model is evaluated for its ability to correct biases on various sets of synthetic data, where the corrected weights lead to improved predictive accuracy across a range of data characteristics. Codes are available at https://github.com/VeronikaWachslander/shrinkage-tuning-bias .