Transforming a Non-Differentiable Rasterizer into a Differentiable One with Stochastic Gradient Estimation
Abstract
We show how to transform a non-differentiable rasterizer into a differentiable one with minimal engineering efforts and no automatic differentiation. To do so, we improve on *Stochastic Gradient Estimation* by using a *Per-Pixel Loss* which leverage the fact that only a few primitives contribute to a given pixel. Estimating gradients on a per-pixel basis bounds the dimensionality of the optimization problem and makes the method scalable. To track parameters contributing to a pixel, we use ID- and UV-buffers, which are often already available or trivial to obtain. With these minor modifications, we obtain an in-engine optimizer for 3D assets with millions of geometry and texture parameters.
Cite
Text
Deliot et al. "Transforming a Non-Differentiable Rasterizer into a Differentiable One with Stochastic Gradient Estimation." ICML 2024 Workshops: Differentiable_Almost_Everything, 2024.Markdown
[Deliot et al. "Transforming a Non-Differentiable Rasterizer into a Differentiable One with Stochastic Gradient Estimation." ICML 2024 Workshops: Differentiable_Almost_Everything, 2024.](https://mlanthology.org/icmlw/2024/deliot2024icmlw-transforming/)BibTeX
@inproceedings{deliot2024icmlw-transforming,
title = {{Transforming a Non-Differentiable Rasterizer into a Differentiable One with Stochastic Gradient Estimation}},
author = {Deliot, Thomas and Heitz, Eric and Belcour, Laurent},
booktitle = {ICML 2024 Workshops: Differentiable_Almost_Everything},
year = {2024},
url = {https://mlanthology.org/icmlw/2024/deliot2024icmlw-transforming/}
}