We introduce some improvements to the Dynamic Learning Vector Quantization algorithm proposed by us for tackling the two major problems of those networks, namely neuron over-splitting and their distribution in the feature space. We suggest to explicitly estimate the potential improvement on the recognition rate achievable by splitting neurons in those regions of the feature space in which two or more classes overlap. We also suggest to compute the neuron splitting frequency, and to combine these information for selecting the most promising neuron to split. Experimental results on both synthetic and real data extracted from UCI Machine Learning Repository show substantial improvements of the proposed algorithm with respect to the state of the art.