Cross-pose facial expression recognition

Abstract

Automatic facial expression recognition is a popular research topic due to its interesting applications in a wide variety of areas. The existing studies have achieved high accuracies in various formulations of the same problem. One direction which is not fully explored is multi-view facial expression recognition. Variations caused by different poses impose extra burden on the task of recognizing expressions, which is already a difficult problem due to large di erences across subjects. In this thesis, we present a method to recognize six prototypic facial expressions of an individual across di erent pose angles. We use Partial Least Squares (PLS) to map the expressions from different poses into a common subspace, in which correlation between them is maximized. Recently, PLS has been successfully used for pose invariant face recognition problem. We show that, PLS can be e ectively used for facial expression recognition across poses by training on coupled expressions of the same identity from two di erent poses. This way of training lets the learned bases model the di erences between expressions of di erent poses by excluding the e ect of the identity. We rst align the faces and then extract block features around two eyes and the mouth on the aligned image. We experiment with Gabor filters and direct intensity values for local face representation. We demonstrate that two representations perform similarly in case frontal is the input pose, but Gabor representation outperforms intensity representation for other pose pairs. We also perform a detailed analysis of the parameters used in the experiments to show their effects on the results and to find the optimal ones for the expression recognition problem.