Consensus of Non-Rigid Reconstructions

• Minsik Lee, Jungchan Cho, and Songhwai Oh, “Consensus of Non-Rigid Reconstructions,” IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2016. (ORAL) [paper | talk | poster | supplementary]

Recently, there have been many progresses for the problem of non-rigid structure reconstruction based on 2D trajectories, but it is still challenging to deal with complex deformations or restricted view ranges. Promising alternatives are the piecewise reconstruction approaches, which divide trajectories into several local parts and stitch their individual reconstructions to produce an entire 3D structure. These methods show the state-of-the-art performance, however, most of them are specialized for relatively smooth surfaces and some are quite complicated. Meanwhile, it has been reported numerously in the field of pattern recognition that obtaining consensus from many weak hypotheses can give a strong, powerful result. Inspired by these reports, in this paper, we push the concept of part-based reconstruction to the limit: Instead of considering the parts as explicitly-divided local patches, we draw a large number of small random trajectory sets. From their individual reconstructions, we pull out a statistic of each 3D point to retrieve a strong reconstruction, of which the procedure can be expressed as a sparse l1-norm minimization problem. In order to resolve the reflection ambiguity between weak (and possibly bad) reconstructions, we propose a novel optimization framework which only involves a single eigenvalue decomposition. The proposed method can be applied to any type of data and outperforms the existing methods for the benchmark sequences, even though it is composed of a few, simple steps. Furthermore, it is easily parallelizable, which is another advantage.

(Data sets are from [White et al., SIGGRAPH 2007, Taylor et al., CVPR 2010, Russell et al., CVPR 2011, Agudo and Moreno-Noguer, ICCV 2015]. We’d like to thank Dr. Antonio Agudo for providing the optical flow result of the back sequence.)

• Matlab code for NRSfM_Consensus