Although handwritten text recognition has been studied for some years, only few authors have used statistical language models to increase the performance of their recognizers. In those few cases where a language model has been used, its integration has not been systematically optimized. In this paper we investigate the optimization of the integration of statistical language models into HMM based recognition systems for offline handwritten text. Based on experiments with the IAM database we show that the recognition performance of a general offline handwritten text recognizer can be substantially improved.