Lagrangian Proximal Gradient Descent for Learning Convex Optimization Models
Abstract
We propose Lagrangian Proximal Gradient Descent (LPGD), a flexible framework for learning convex optimization models. Like traditional proximal gradient methods, LPGD can be interpreted as optimizing a smoothed envelope of the possibly non-differentiable loss. The smoothening allows training models that do not provide informative gradients, such as discrete optimization models. We show that the LPGD update can efficiently be computed by rerunning the forward solver on a perturbed input. Moreover, we prove that the LPGD update converges to the gradient as the smoothening parameter approaches zero. Finally, we experimentally investigate the potential benefits of applying LPGD even in a fully differentiable setting.
Cite
Text
Paulus et al. "Lagrangian Proximal Gradient Descent for Learning Convex Optimization Models." ICML 2023 Workshops: Differentiable_Almost_Everything, 2023.Markdown
[Paulus et al. "Lagrangian Proximal Gradient Descent for Learning Convex Optimization Models." ICML 2023 Workshops: Differentiable_Almost_Everything, 2023.](https://mlanthology.org/icmlw/2023/paulus2023icmlw-lagrangian/)BibTeX
@inproceedings{paulus2023icmlw-lagrangian,
title = {{Lagrangian Proximal Gradient Descent for Learning Convex Optimization Models}},
author = {Paulus, Anselm and Musil, Vít and Martius, Georg},
booktitle = {ICML 2023 Workshops: Differentiable_Almost_Everything},
year = {2023},
url = {https://mlanthology.org/icmlw/2023/paulus2023icmlw-lagrangian/}
}