In this work an image retrieval system adaptable to user's interests by the use of relevance feedback via genetic algorithm is presented. The retrieval process is based on local similarity patterns. The goal of the genetic algorithm is to infer weights for regions and features that better trans- late the user's requirements producing better quality rank- ings. The genetic algorithm used has as its main innovation an order-based fitness function, which is appropriate to the ranking requirements of a majority of the users. This fitness function will quickly drive the genetic algorithm in the pro- cess of searching for an optimal solution. Evaluations in several databases have shown the robustness and efficiency of the proposed retrieval method even when the query is a sketch or damaged image.