KAFÈ: Kernel Aggregation for FEderated

Abstract

The convergence and generalization abilities of federated learning (FL) models encounter significant obstacles when confronted with non-independent and identically distributed (non-IID) data. This situation leads to wandering behaviors among individual clients, thereby causing the global model to deviate from local optimal states. Recent research has shed light on this phenomenon, indicating that it may be attributed to biases introduced by local updates from clients. Specifically, it has been found that the primary reason for the performance degradation of model trained on non-IID data is the classifier, as biased local classifiers struggle to effectively classify unseen data due to shifted decision boundaries influenced by local class distributions. Motivated by the above insight, we present KAFÈ , a novel yet simple aggregation solution designed to address non-IID-ness by introducing K ernel A ggregation for FE derated Learning. KAFÈ leverages Kernel Density Estimation (KDE) to construct a novel classification layer for the global model, drawing upon the estimated weight distributions of the individual classifiers. We conducted several experiments on image and text datasets to evaluate KAFÈ’s performance, consistently observing superior results compared to other state-of-the-art (SOTA) FL methods. Notably, KAFÈ achieves these results with reduced computational overhead compared to leading SOTA approaches. Full code can be accessed at: https://github.com/MODAL-UNINA/KAFE .