Investigating the Impact of Missing Value Handling on Boosted Trees and Deep Learning for Tabular Data: A Claim Reserving Case Study

Abstract

While deep learning (DL) performance is exceptional for many applications, there is no consensus on whether DL or gradient boosted decision trees (GBDTs) are superior for tabular data. We compare TabNet (a DL model for tabular data), two simple neural networks inspired by ResNet (a DL model) and Catboost (a GBDT model) on a large UK insurer dataset for the task of claim reserving. This dataset is of particular interest for its large amount of informative missing values which are not missing completely at random, highlighting the impact of missing value handling on accuracy. Under certain missing value schemes a carefully optimised simple neural network performed comparably to Catboost with default settings. However, using less-than-minimum imputation, Catboost with default settings substantially outperformed carefully optimised DL models, achieving the best overall accuracy. We conclude that handling missing values is an important, yet often overlooked, step when comparing DL to GBDT algorithms for tabular data.