The Differential Geometry of Shape from Shading: Biology Reveals Curvature Structure

Abstract

Shape from shading is a classical inverse problem in computer vision. We introduce a novel mathematical formulation for calculating local surface shape based on covariant derivatives, rather than the customary integral minimization or P.D.E approaches. Motivated by neurobiology, we introduce the shading flow field (the tangent map to the image isophotes) between the image and the surface levels. Just as in the perceptual organization of texture, we use the parallel transport of our shading flow field to move the isophote field at different points on the unknown surface to a single point, amassing restrictions on our surface curvatures. Under simplifying assumptions we solve exactly for the light source/surface pairs needed for a local image patch to have a given shading flow. The magnitude of the brightness gradient then restricts this family to a single light source and surface estimate pair, up to the concave/convex ambiguity and an additional elliptical/saddle ambiguity. Example calculations illustrate our approach.