A recent thrust of non-cooperative target recognition (NCTR) using synthetic aperture radar (SAR) has been to complement the extraction of scattering centers by incorporating information contained in the target shadow. When classifying targets based on the shadow region alone, it is essential that an image be well clustered into its respective shadow, highlight, and background regions. To obtain the segmentation, the intensity and spatial location of a pixel are modeled as a mixture of Gaussian distributions. Expectation-maximization (EM) is used to obtain the corresponding distributions for the three regions within a given image. Anisotropic smoothing is applied to smooth the input image as well as the posterior probabilities. A representation of the shadow boundary is developed in conjunction with a Hidden Markov Model (HMM) ensemble to obtain target classification. A variety of targets from the MSTAR database are used to test the performance of both the segmentation algorithm and classification structure.