An Overview of Recent CMU Research on Model-Based Face Processing

A face model is a mapping from a set of parameters to an image of a face. The most well-known face models are Active Appearance Models and 3D Morphable Models. Computer vision applications of face models include head pose estimation for user interfaces, gaze estimation, pose normalization for face recognition, lip-reading, expression recognition, and face coding for low-bandwidth video-conferencing. In all of these applications, the key task is to fit the face model to an input image; i.e. to find the parameters of the model that match the input image as well as possible. Applying model fitting to each image in a video in turn results in a non-rigid face tracking algorithm.

In this talk I will describe how face model fitting, a non-linear optimization, can be posed as an image alignment problem. Image alignment is a standard computer vision technique, with applications to optical flow, tracking, mosaic construction, layered scene representations, and medical image registration. I will describe a new efficient image alignment algorithm and show how it relates to others in a unifying framework. Applying our algorithm to faces results in real-time 2D, 3D, and multi-view face model fitting algorithms.

I will also describe some of our recent research on face model construction, including automatic (unsupervised) model construction, model update, and 3D model construction from 2D images.