Shape-from-Polarisation: A Nonlinear Least Squares Approach

Abstract

In this paper we present a new type of approach for estimating surface height from polarimetric data, i.e. a sequence of images in which a linear polarising filter is rotated in front of a camera. In contrast to all previous shape-from-polarisation methods, we do not first transform the observed data into a polarisation image. Instead, we minimise the sum of squared residuals between predicted and observed intensities over all pixels and polariser angles. This is a nonlinear least squares optimisation problem in which the unknown is the surface height. The forward prediction is a series of transformations for which we provide analytical derivatives allowing the overall problem to be efficiently optimised using Gauss-Newton type methods with an analytical Jacobian matrix. The method is very general and can incorporate any (differentiable) illumination, reflectance or polarisation model. We also propose a variant of the method which uses image ratios to remove dependence on illumination and albedo. We demonstrate our methods on glossy objects, including with albedo variations, and provide comparison to a state of the art approach.