Material Swapping for 3D Scenes Using a Learnt Material Similarity Measure
Abstract
We present a method for augmenting photo-realistic 3D scene assets by automatically recognizing, matching, and swapping their materials. Our method proposes a material matching pipeline for the efficient replacement of unknown materials with perceptually similar PBR materials from a database, enabling the quick creation of many variations of a given 3D synthetic scene. At the heart of this method is a novel material similarity feature that is learnt, in conjunction with optimal lighting conditions, by fine-tuning a deep neural network on a material classification task using our proposed dataset. Our evaluation demonstrates that lighting optimization improves CNN-based texture feature extraction methods and better estimates material properties. We conduct a series of experiments showing our method's ability to augment photo-realistic indoor scenes using both standard and procedurally generated PBR materials.
Cite
Text
Perroni-Scharf et al. "Material Swapping for 3D Scenes Using a Learnt Material Similarity Measure." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2022. doi:10.1109/CVPRW56347.2022.00221Markdown
[Perroni-Scharf et al. "Material Swapping for 3D Scenes Using a Learnt Material Similarity Measure." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2022.](https://mlanthology.org/cvprw/2022/perronischarf2022cvprw-material/) doi:10.1109/CVPRW56347.2022.00221BibTeX
@inproceedings{perronischarf2022cvprw-material,
title = {{Material Swapping for 3D Scenes Using a Learnt Material Similarity Measure}},
author = {Perroni-Scharf, Maxine and Sunkavalli, Kalyan and Eisenmann, Jonathan and Hold-Geoffroy, Yannick},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
year = {2022},
pages = {2033-2042},
doi = {10.1109/CVPRW56347.2022.00221},
url = {https://mlanthology.org/cvprw/2022/perronischarf2022cvprw-material/}
}