K-means Algorithm is a popular method in cluster anal- ysis. After reviewing different K-means algorithms, we pro- pose the new penalized K-means algorithm. Originally in- spired by the Maximum Likelihood(ML) method, a prior probability distribution assumed by classic K-means algo- rithm about the clustering data set was discovered, and then the new objective function for the penalized K-means algo- rithm was introduced. By minimizing this function with ge- netic algorithm, results show that this method is better than K-means algorithm in some perspectives.