Alchemist: Parametric Control of Material Properties with Diffusion Models
Abstract
We propose a method to control material attributes of objects like roughness metallic albedo and transparency in real images. Our method capitalizes on the generative prior of text-to-image models known for photorealism employing a scalar value and instructions to alter low-level material properties. Addressing the lack of datasets with controlled material attributes we generated an object-centric synthetic dataset with physically-based materials. Fine-tuning a modified pre-trained text-to-image model on this synthetic dataset enables us to edit material properties in real-world images while preserving all other attributes. We show the potential application of our model to material edited NeRFs.
Cite
Text
Sharma et al. "Alchemist: Parametric Control of Material Properties with Diffusion Models." Conference on Computer Vision and Pattern Recognition, 2024. doi:10.1109/CVPR52733.2024.02278Markdown
[Sharma et al. "Alchemist: Parametric Control of Material Properties with Diffusion Models." Conference on Computer Vision and Pattern Recognition, 2024.](https://mlanthology.org/cvpr/2024/sharma2024cvpr-alchemist/) doi:10.1109/CVPR52733.2024.02278BibTeX
@inproceedings{sharma2024cvpr-alchemist,
title = {{Alchemist: Parametric Control of Material Properties with Diffusion Models}},
author = {Sharma, Prafull and Jampani, Varun and Li, Yuanzhen and Jia, Xuhui and Lagun, Dmitry and Durand, Fredo and Freeman, Bill and Matthews, Mark},
booktitle = {Conference on Computer Vision and Pattern Recognition},
year = {2024},
pages = {24130-24141},
doi = {10.1109/CVPR52733.2024.02278},
url = {https://mlanthology.org/cvpr/2024/sharma2024cvpr-alchemist/}
}