Scotopic Visual Recognition

Abstract

Recognition from a small number of photons is important for biomedical imaging, security, astronomy and many other fields. We develop a framework that allows a machine to classify objects as quickly as possible, hence requiring as few photons as possible, while maintaining the error rate below an acceptable threshold. The framework also allows for a dynamic speed versus accuracy tradeoff. Given a generative model of the scene, the optimal tradeoff can be obtained from a self-recurrent deep neural network. The generative model may also be learned from the data. We find that MNIST classification performance from less than 1 photon per pixel is comparable to that obtained from images in normal lighting conditions. Classification on CIFAR10 requires 10 photon per pixel to stay within 1% the normal-light performance.