$f$-SCRUB: Unbounded Machine Unlearning via $f$-Divergences

Abstract

Deep Machine Unlearning addresses the problem of removing the effect of a subset of data points from a trained model. Machine Unlearning has various implications for the performance of algorithms. A well-known algorithm, SCRUB~\citep{kurmanji2023unboundedmachineunlearning}, has served as a baseline and achieved key objectives such as removing biases, resolving confusion caused by mislabeled data in trained models, and allowing users to exercise their "right to be forgotten" to protect user privacy. Building on this algorithm, we introduce $f$-SCRUB, an extension of SCRUB that employs different $f$-divergences instead of KL divergence. We analyze the role of these divergences and their impact on the resolution of unlearning problems in various scenarios.