Sensor-Realistic Synthetic Data Engine for Multi-Frame High Dynamic Range Photography

Jinhan Hu, Gyeongmin Choe, Zeeshan Nadir, Osama Nabil, Seok-Jun Lee, Hamid R. Sheikh, Youngjun Francis Yoo, Michael O. Polley

CVPRW 2020 pp. 2180-2189

doi:10.1109/CVPRW50498.2020.00266 /cvprw/2020/hu2020cvprw-sensorrealistic/

Abstract

Deep learning-based mobile imaging applications are often limited by the lack of training data. To this end, researchers have resorted to using synthetic training data. However, pure synthetic data does not accurately mimic the distribution of the real data. To improve the utility of synthetic data, we present a systematic pipeline that takes synthetic data coming purely from a game engine and then produces synthetic data with real sensor characteristics such as noise and color gamut. We validate the utility of our sensor-realistic synthetic data for multi-frame high dynamic range (HDR) photography using a Samsung Galaxy S10 Plus smartphone. The result of training two baseline neural networks using our sensor realistic synthetic data modeled for the S10 Plus show that our sensor realistic synthetic data improves the quality of HDR photography on the modeled device. The synthetic dataset is publicly available at https://github.com/nadir-zeeshan/sensor-realistic-synthetic-data.

CVPRW Semantic Scholar

Cite

Text

Hu et al. "Sensor-Realistic Synthetic Data Engine for Multi-Frame High Dynamic Range Photography." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020. doi:10.1109/CVPRW50498.2020.00266

Markdown

[Hu et al. "Sensor-Realistic Synthetic Data Engine for Multi-Frame High Dynamic Range Photography." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020.](https://mlanthology.org/cvprw/2020/hu2020cvprw-sensorrealistic/) doi:10.1109/CVPRW50498.2020.00266

BibTeX

@inproceedings{hu2020cvprw-sensorrealistic,
  title     = {{Sensor-Realistic Synthetic Data Engine for Multi-Frame High Dynamic Range Photography}},
  author    = {Hu, Jinhan and Choe, Gyeongmin and Nadir, Zeeshan and Nabil, Osama and Lee, Seok-Jun and Sheikh, Hamid R. and Yoo, Youngjun Francis and Polley, Michael O.},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
  year      = {2020},
  pages     = {2180-2189},
  doi       = {10.1109/CVPRW50498.2020.00266},
  url       = {https://mlanthology.org/cvprw/2020/hu2020cvprw-sensorrealistic/}
}