PhySG: Inverse Rendering with Spherical Gaussians for Physics-Based Material Editing and Relighting
Abstract
We present an end-to-end inverse rendering pipeline that includes a fully differentiable renderer, and can reconstruct geometry, materials, and illumination from scratch from a set of images. Our rendering framework represents specular BRDFs and environmental illumination using mixtures of spherical Gaussians, and represents geometry as a signed distance function parameterized as a Multi-Layer Perceptron. The use of spherical Gaussians allows us to efficiently solve for approximate light transport, and our method works on scenes with challenging non-Lambertian reflectance captured under natural, static illumination. We demonstrate, with both synthetic and real data, that our reconstruction not only can render novel viewpoints, but also enables physics-based appearance editing of materials and illumination.
Cite
Text
Zhang et al. "PhySG: Inverse Rendering with Spherical Gaussians for Physics-Based Material Editing and Relighting." Conference on Computer Vision and Pattern Recognition, 2021. doi:10.1109/CVPR46437.2021.00541Markdown
[Zhang et al. "PhySG: Inverse Rendering with Spherical Gaussians for Physics-Based Material Editing and Relighting." Conference on Computer Vision and Pattern Recognition, 2021.](https://mlanthology.org/cvpr/2021/zhang2021cvpr-physg/) doi:10.1109/CVPR46437.2021.00541BibTeX
@inproceedings{zhang2021cvpr-physg,
title = {{PhySG: Inverse Rendering with Spherical Gaussians for Physics-Based Material Editing and Relighting}},
author = {Zhang, Kai and Luan, Fujun and Wang, Qianqian and Bala, Kavita and Snavely, Noah},
booktitle = {Conference on Computer Vision and Pattern Recognition},
year = {2021},
pages = {5453-5462},
doi = {10.1109/CVPR46437.2021.00541},
url = {https://mlanthology.org/cvpr/2021/zhang2021cvpr-physg/}
}