A novel indexing and access method, called Affinity Hybrid Tree (AH-Tree), is proposed to organize large image data sets efficiently and to support popular image access mechanisms like Content-Based Image Retrieval (CBIR) by embedding the high-level semantic image-relationship in the access mechanism as it is. AH-Tree combines Space- Based and Distance-Based indexing techniques to form a hybrid structure which is efficient in terms of computational overhead and fairly accurate in producing query results close to human perception. Algorithms for similarity (range and k-nearest neighbor) queries are implemented. Results from elaborate experiments are reported which depict a low computational overhead in terms of the number of I/O and distance computations and a high relevance of query results. The proposed index structure solves the existing problems of introducing high-level image relationships in a retrieval mechanism without going through the pain of translating the content-similarity measurement into feature-level equivalence and yet maintaining an efficient structure to organize the large sets of images.