Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report
Abstract
The role of mobile cameras increased dramatically over the past few years, leading to more and more research in automatic image quality enhancement and RAW photo processing. In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based image signal processing (ISP) pipeline replacing the standard mobile ISPs that can run on modern smartphone GPUs using TensorFlow Lite. The participants were provided with a large-scale Fujifilm UltraISP dataset consisting of thousands of paired photos captured with a normal mobile camera sensor and a professional 102MP medium-format FujiFilm GFX100 camera. The runtime of the resulting models was evaluated on the Snapdragon’s 8 Gen 1 GPU that provides excellent acceleration results for the majority of common deep learning ops. The proposed solutions are compatible with all recent mobile GPUs, being able to process Full HD photos in less than 20–50 ms while achieving high fidelity results. A detailed description of all models developed in this challenge is provided in this paper.
Cite
Text
Ignatov et al. "Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report." European Conference on Computer Vision Workshops, 2022. doi:10.1007/978-3-031-25066-8_3Markdown
[Ignatov et al. "Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report." European Conference on Computer Vision Workshops, 2022.](https://mlanthology.org/eccvw/2022/ignatov2022eccvw-learned/) doi:10.1007/978-3-031-25066-8_3BibTeX
@inproceedings{ignatov2022eccvw-learned,
title = {{Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report}},
author = {Ignatov, Andrey and Timofte, Radu and Liu, Shuai and Feng, Chaoyu and Bai, Furui and Wang, Xiaotao and Lei, Lei and Yi, Ziyao and Xiang, Yan and Liu, Zibin and Li, Shaoqing and Shi, Keming and Kong, Dehui and Xu, Ke and Kwon, Minsu and Wu, Yaqi and Zheng, Jiesi and Fan, Zhihao and Wu, Xun and Zhang, Feng and No, Albert and Cho, Minhyeok and Chen, Zewen and Zhang, Xiaze and Li, Ran and Wang, Juan and Wang, Zhiming and Conde, Marcos V. and Choi, Ui-Jin and Perevozchikov, Georgy and Ershov, Egor I. and Hui, Zheng and Dong, Mengchuan and Lou, Xin and Zhou, Wei and Pang, Cong and Qin, Haina and Cai, Mingxuan},
booktitle = {European Conference on Computer Vision Workshops},
year = {2022},
pages = {44-70},
doi = {10.1007/978-3-031-25066-8_3},
url = {https://mlanthology.org/eccvw/2022/ignatov2022eccvw-learned/}
}