Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining
Abstract
The performance of differentially private machine learning can be boosted significantly by leveraging the transfer learning capabilities of non-private models pretrained on large public datasets. We critically review this approach. We primarily question whether the use of large Web-scraped datasets should be viewed as differential-privacy-preserving. We further scrutinize whether existing machine learning benchmarks are appropriate for measuring the ability of pretrained models to generalize to sensitive domains. Finally, we observe that reliance on large pretrained models may lose other forms of privacy, requiring data to be outsourced to a more compute-powerful third party.
Cite
Text
Tramèr et al. "Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining." International Conference on Machine Learning, 2024.Markdown
[Tramèr et al. "Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining." International Conference on Machine Learning, 2024.](https://mlanthology.org/icml/2024/tramer2024icml-position/)BibTeX
@inproceedings{tramer2024icml-position,
title = {{Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining}},
author = {Tramèr, Florian and Kamath, Gautam and Carlini, Nicholas},
booktitle = {International Conference on Machine Learning},
year = {2024},
pages = {48453-48467},
volume = {235},
url = {https://mlanthology.org/icml/2024/tramer2024icml-position/}
}