Abstract: We propose a novel representation for characterizing image differences using a deformable technique for obtaining pixel-wise correspondences. This representation, which is based on a deformable 3D mesh in XYI-space, is then experimentally compared with two related correspondence methods: optical flow and intensity differences. Furthermore, we make use of a probabilistic similarity measure for direct image matching based on a Bayesian analysis of image variations. We model two classes of variation in facial appearance: intra-personal and extra-personal. The probability density function for each class is estimated from training data and used to compute a similarity measure based on the a posteriori probabilities. The performance advantage of our deformable probabilistic matching technique is demonstrated using 1700 faces from the US Army's "FERET" face database.