Differential Geometry Boosts Convolutional Neural Networks for Object Detection

Abstract

Convolutional neural networks (CNNs) have had dramatic success in appearance based object recognition tasks such as the ImageNet visual recognition challenge [8]. However, their application to object recognition and detection thus far has focused largely on intensity or color images as inputs. Motivated by demonstrations that depth can enhance the performance of CNN-based approaches [2][5], in this paper we consider the benefits of also including differential geometric shape features. This elementary idea of using zeroth order (depth), first-order (surface normal) and second-order (surface curvature) features in a principled manner boosts the performance of a CNN that has been pretrained on a color image database. Notably, in an object detection task involving 19 categories we achieve 39.30% accuracy on the NYUv2 dataset, which is a 10.4% improvement over the current state-of-the-art accuracy of 35.6% using the method in [5]. In the simpler scenario of turntable style object recognition, our experiments on the University of Washington (UW) RGB-D dataset yield an accuracy of 88.7% correct recognition over 51 object categories, where the best competing result is 87.5% [2]. Taken together, our results provide strong evidence that the abstraction of surface shape benefits object detection and recognition.