Ray Conditioning: Trading Photo-Consistency for Photo-Realism in Multi-View Image Generation
Abstract
Multi-view image generation attracts particular attention these days due to its promising 3D-related applications, e.g., image viewpoint editing. Most existing methods follow a paradigm where a 3D representation is first synthesized, and then rendered into 2D images to ensure photo-consistency across viewpoints. However, such explicit bias for photo-consistency sacrifices photo-realism, causing geometry artifacts and loss of fine-scale details when these methods are applied to edit real images. To address this issue, we propose ray conditioning, a geometry-free alternative that relaxes the photo-consistency constraint. Our method generates multi-view images by conditioning a 2D GAN on a light field prior. With explicit viewpoint control, state-of-the-art photo-realism and identity consistency, our method is particularly suited for the viewpoint editing task.
Cite
Text
Chen et al. "Ray Conditioning: Trading Photo-Consistency for Photo-Realism in Multi-View Image Generation." International Conference on Computer Vision, 2023. doi:10.1109/ICCV51070.2023.02124Markdown
[Chen et al. "Ray Conditioning: Trading Photo-Consistency for Photo-Realism in Multi-View Image Generation." International Conference on Computer Vision, 2023.](https://mlanthology.org/iccv/2023/chen2023iccv-ray/) doi:10.1109/ICCV51070.2023.02124BibTeX
@inproceedings{chen2023iccv-ray,
title = {{Ray Conditioning: Trading Photo-Consistency for Photo-Realism in Multi-View Image Generation}},
author = {Chen, Eric Ming and Holalkere, Sidhanth and Yan, Ruyu and Zhang, Kai and Davis, Abe},
booktitle = {International Conference on Computer Vision},
year = {2023},
pages = {23242-23251},
doi = {10.1109/ICCV51070.2023.02124},
url = {https://mlanthology.org/iccv/2023/chen2023iccv-ray/}
}