High-Resolution Modeling of Moving and Deforming Objects Using Sparse Geometric and Dense Photometric Measurements

Abstract

Modeling moving and deforming objects requires capturing as much information as possible during a very short time. When using off-the-shelf hardware, this often hinders the resolution and accuracy of the acquired model. Our key observation is that in as little as four frames both sparse surface-positional measurements and dense surface-orientation measurements can be acquired using a combination of structured light and photometric stereo, resulting in high-resolution models of moving and deforming objects. Our system projects alternating geometric and photometric patterns onto the object using a set of three projectors and captures the object using a synchronized camera. Small motion among temporally close frames is compensated by estimating the optical flow of images captured under the uniform illumination of the photometric light. Then spatial-temporal photogeometric reconstructions are performed to obtain dense and accurate point samples with a sampling resolution equal to that of the camera. Temporal coherence is also enforced. We demonstrate our system by successfully modeling several moving and deforming real-world objects.