Handwriting recognition and OCR systems need to cope with a wide variety of writing styles and fonts, many of them possibly not previously encountered during training. This paper describes a notion of Bayesian statistical similarity and demonstrates how it can be applied to rapid adaptation to new styles. The ability to generalize across different problem instances is illustrated in the Gaussian case, and the use of statistical similarity Gaussian case is shown to be related to adaptive metric classification methods. The relationship to prior approaches to multitask learning, as well as variable or adaptive metric classification, and hierarchical Bayesian methods, are discussed. Experimental results on character recognition from the NIST3 database are presented.