No Shadow Left Behind: Removing Objects and Their Shadows Using Approximate Lighting and Geometry
Abstract
Removing objects from images is a challenging technical problem that is important for many applications, including mixed reality. For believable results, the shadows that the object casts should also be removed. Current inpainting-based methods only remove the object itself, leaving shadows behind, or at best require specifying shadow regions to inpaint. We introduce a deep learning pipeline for removing a shadow along with its caster. We leverage rough scene models in order to remove a wide variety of shadows (hard or soft, dark or subtle, large or thin) from surfaces with a wide variety of textures. We train our pipeline on synthetically rendered data, and show qualitative and quantitative results on both synthetic and real scenes.
Cite
Text
Zhang et al. "No Shadow Left Behind: Removing Objects and Their Shadows Using Approximate Lighting and Geometry." Conference on Computer Vision and Pattern Recognition, 2021. doi:10.1109/CVPR46437.2021.01613Markdown
[Zhang et al. "No Shadow Left Behind: Removing Objects and Their Shadows Using Approximate Lighting and Geometry." Conference on Computer Vision and Pattern Recognition, 2021.](https://mlanthology.org/cvpr/2021/zhang2021cvpr-shadow/) doi:10.1109/CVPR46437.2021.01613BibTeX
@inproceedings{zhang2021cvpr-shadow,
title = {{No Shadow Left Behind: Removing Objects and Their Shadows Using Approximate Lighting and Geometry}},
author = {Zhang, Edward and Martin-Brualla, Ricardo and Kontkanen, Janne and Curless, Brian L.},
booktitle = {Conference on Computer Vision and Pattern Recognition},
year = {2021},
pages = {16397-16406},
doi = {10.1109/CVPR46437.2021.01613},
url = {https://mlanthology.org/cvpr/2021/zhang2021cvpr-shadow/}
}