Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations
Abstract
We describe a procedure for removing dependency on a cohort of training data from a trained deep network that improves upon and generalizes previous methods to different readout functions, and can be extended to ensure forgetting in the final activations of the network. We introduce a new bound on how much information can be extracted per query about the forgotten cohort from a black-box network for which only the input-output behavior is observed. The proposed forgetting procedure has a deterministic part derived from the differential equations of a linearized version of the model, and a stochastic part that ensures information destruction by adding noise tailored to the geometry of the loss landscape. We exploit the connections between the final activations and weight dynamics of a DNN inspired by Neural Tangent Kernels to compute the information in the final activations.
Cite
Text
Golatkar et al. "Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations." Proceedings of the European Conference on Computer Vision (ECCV), 2020. doi:10.1007/978-3-030-58526-6_23Markdown
[Golatkar et al. "Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations." Proceedings of the European Conference on Computer Vision (ECCV), 2020.](https://mlanthology.org/eccv/2020/golatkar2020eccv-forgetting/) doi:10.1007/978-3-030-58526-6_23BibTeX
@inproceedings{golatkar2020eccv-forgetting,
title = {{Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations}},
author = {Golatkar, Aditya and Achille, Alessandro and Soatto, Stefano},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
year = {2020},
doi = {10.1007/978-3-030-58526-6_23},
url = {https://mlanthology.org/eccv/2020/golatkar2020eccv-forgetting/}
}