In this paper we report the use of discriminative training and other techniques to improve performance in a HMM-based isolated handwritten character recognition system. The discriminative training is Maximum Mutual Information (MMI) training; we also improve results by using composite images which are the concatenation of the raw images, rotated and polar transformed versions of them; and we describe a technique called block-based Principal Component Analysis (PCA). For effective discriminative training we need to increase the size of our training database, which we do by eroding and dilating the images to give a three-fold increase in training data. Although these techniques are tested using isolated Thai characters, both MMI and block-based PCA are applicable to the more difficult task of cursive handwriting recognition.