An optimal frequency domain bit allocation technique is presented. A Lagrange multiplier based approach is used for the rate-distortion optimization of the quantization matrix in MPEG-2 encoding. Since the optimal quantization matrix is heavily dependent on the choice of the quantizer, the quantizer is also jointly optimized in order to alleviate the interaction between the two. The complexity of the algorithm is analyzed, and it is shown to be directly proportional to the number of quantization matrices and quantization scales. Experimental results comparing the new technique with a conventional MPEG-2 encoding are also presented