In this paper we present a bit allocation approach based on motion vector analysis for improved Rate-Distortion performance. Bit allocation is done at the macroblock level such that the macroblock with high priority is coded finely and the one with low priority is coded coarsely. In order to calculate macroblock priorities, first reference counts for each pixel are determined through motion vector analysis. A reference count of a pixel is defined as total number of pixels in the remaining GOP which use that pixel as a reference. Then macroblock wise reference counts is obtained by summing the pixel wise reference counts, which are then scaled and prioritized. Based upon the priority value the given fixed quant of each frame is modulated at the macroblock level. The algorithm is applied to H.264/AVC encoding and PSNR gains of up to 1.4 dB are achieved.