Floors Are Flat: Leveraging Semantics for Real-Time Surface Normal Prediction
Abstract
We propose 4 insights that help to significantly improve the performance of deep learning models that predict surface normals and semantic labels from a single RGB image. These insights are: (1) denoise the "ground truth" surface normals in the training set to ensure consistency with the semantic labels; (2) concurrently train on a mix of real and synthetic data, instead of pretraining on synthetic and finetuning on real; (3) jointly predict normals and semantics using a shared model, but only backpropagate errors on pixels that have valid training labels; (4) slim down the model and use grayscale instead of color inputs. Despite the simplicity of these steps, we demonstrate consistently improved state of the art results on several datasets, using a model that runs at 12 fps on a standard mobile phone.
Cite
Text
Hickson et al. "Floors Are Flat: Leveraging Semantics for Real-Time Surface Normal Prediction." IEEE/CVF International Conference on Computer Vision Workshops, 2019. doi:10.1109/ICCVW.2019.00501Markdown
[Hickson et al. "Floors Are Flat: Leveraging Semantics for Real-Time Surface Normal Prediction." IEEE/CVF International Conference on Computer Vision Workshops, 2019.](https://mlanthology.org/iccvw/2019/hickson2019iccvw-floors/) doi:10.1109/ICCVW.2019.00501BibTeX
@inproceedings{hickson2019iccvw-floors,
title = {{Floors Are Flat: Leveraging Semantics for Real-Time Surface Normal Prediction}},
author = {Hickson, Steven and Raveendran, Karthik and Fathi, Alireza and Murphy, Kevin and Essa, Irfan A.},
booktitle = {IEEE/CVF International Conference on Computer Vision Workshops},
year = {2019},
pages = {4065-4074},
doi = {10.1109/ICCVW.2019.00501},
url = {https://mlanthology.org/iccvw/2019/hickson2019iccvw-floors/}
}