Facial Shape Recovery from a Single Image under General, Unknown Lighting

• Minsik Lee and Chong-Ho Choi, “Real-Time Facial Shape Recovery from a Single Image under General, Unknown Lighting by Rank Relaxation,” Computer Vision and Image Understanding, vol. 120, pp. 59-69, Mar. 2014. [paper]
• Minsik Lee and Chong-Ho Choi, “A Robust Real-Time Algorithm for Facial Shape Recovery from a Single Image Containing Cast Shadow under General, Unknown Lighting,” Pattern Recognition, vol. 46, no. 1, pp. 38-44, Jan. 2013. [paper]
• Minsik Lee and Chong-Ho Choi, “Fast Facial Shape Recovery From a Single Image with General, Unknown Lighting by Using Tensor Representation,” Pattern Recognition, vol. 44, no. 7, pp. 1487-1496, Jul. 2011. [paper]

Statistical shape from shading under general light conditions can be thought of as a parameter-fitting problem to a bilinear model. Here, the parameters are personal attributes and light conditions. Parameters of a bilinear model are usually estimated using the alternating least squares method with a computational complexity of O((n_s+n_phi)^2 n_p), where n_s, n_phi, and n_p are the dimensions of the light conditions, personal attributes, and face image features, respectively, for each iteration. In this paper, we propose an alternative algorithm with a computational complexity of O(n_s n_phi) for each iteration. Only the initial step requires a computational complexity of O(n_s n_phi n_p). This can be accomplished by reformulating the problem to that of a linear least squares problem, with a search space limited to a set of rank-one matrices. The rank-one condition is relaxed to obtain a possibly full-rank matrix. The algorithm then finds the best rank-one approximation of that matrix. By the Eckart-Young theorem, the best approximation is the outer product of the left and right singular vectors corresponding to the largest singular value. Since only this pair of singular vectors is needed, it is better to use the power iteration method, which has a computational complexity of O(n_s n_phi) for each iteration, than calculating the full singular value decomposition. The proposed method provides accurate reconstruction results and takes approximately 45 ms on a PC, which is adequate for real-time applications.

(The input images were downloaded from various web pages.)

• Matlab code for testing rank-relaxation method with a prebuilt model
• Matlab code for training and testing all of the methods