We present a technique to adapt the domain of local features through the matching process to augment their discriminative power. We start with local affine features selected and normalized independently in training and test images, and jointly expand their domain as part of the correspondence process, akin to a (non-rigid) registration task that yields a (multi-view) segmentation of the object of interest from clutter, including the detection of occlusions. We show how our growth process can be used to validate putative affine matches, to match a given "template" (an image of an object without clutter) to a cluttered and partially occluded image, and to match two images that contain the same unknown object in different clutter under different occlusions (unsupervised object detection).