Multispectral Direct-Global Separation of Dynamic Scenes

Abstract

In this paper, we propose a method for separating direct and global components of a dynamic scene per illumination color by using a projector-camera system; it exploits both the color switch and the temporal dithering of a DLP projector. Our proposed method is easy-to-implement because it does not require any self-built equipment and temporal synchronization between a projector and a camera. In addition, our method automatically calibrates the projector-camera correspondence in a dynamic scene on the basis of the consistency in pixel intensities, and optimizes the projection pattern on the basis of noise propagation analysis. We implemented the prototype setup and achieved multispectral direct-global separation of dynamic scenes in 60 Hz. Furthermore, we demonstrated that our method is effective for applications such as image-based material editing and multispectral relighting of dynamic scenes where wavelength-dependent phenomena such as fluorescence are observed.