A Unified Framework for Performance Evaluation of 3-D Reconstruction Techniques

Abstract

In this paper we consider the problem of performance evaluation of 3-D reconstruction techniques from calibrated sequence of images. We propose a unified framework for the evaluation process assuming the existence of ground truth data collected by a 3-D laser scanner. Firstly, we propose a new pre-evaluation technique for 3-D data registration based on silhouettes alignment. The performance of this technique is independent of the output quality of the 3-D reconstruction technique under test. This is a major advantage of our technique since the quality issue, which is not guarantee under the evaluation topic, is a challenge for any 3-D registration technique. Secondly, we propose a geometrically based evaluation methodology. The method is based on the geometrical matching of uniformly distributed patches of both ground truth and under-test data sets. The method is insensitive to the relative density of data sets. In addition, it provides a method for localizing errors which is a crucial step for diagnosis and data fusion post-evaluation techniques. Finally, we provide experimental results on the evaluation of stereo and space carving techniques.