loading...
Independent Quantization: An Index Compression Technique for High-Dimensional Data Spaces
San Diego, California February 28-March 03
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/ICDE.2000.83945616th International Conference on Data ...
 This Article 
 
PDF
HTML
 
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Stefan Berchtold, Software Technologie Beratung GMBH
Christian Böhm, Software Technologie Beratung GMBH and University of Munich
Hans-Peter Kriegel, University of Munich
Jörg Sander, University of Munich
H.V. Jagadish, University of Michigan
Two major approaches have been proposed to efficiently process queries in databases: Speeding up the search by using index structures, and speeding up the search by operating on a compressed database, such as a signature file. Both approaches have their limitations: Indexing techniques are inefficient in extreme configurations, such as high-dimensional spaces, where even a simple scan may be cheaper than an index-based search. Compression techniques are not very efficient in all other situations. We propose to combine both techniques to search for nearest neighbors in a high-dimensional space.For this purpose, we develop a compressed index, called the IQ-tree, with a three-level structure: The first level is a regular (flat) directory consisting of minimum bounding boxes, the second level contains data points in a compressed representation, and the third level contains the actual data.We overcome several engineering challenges in constructing an effective index structure of this type. The most significant of these is to decide how much to compress at the second level. Too much compression will lead to many needless expensive accesses to the third level. Too little compression will increase both the storage and the access cost for the first two levels.We develop a cost model and an optimization algorithm based on this cost model that permits an independent determination of the degree of compression for each second level page to minimize expected query cost. In an experimental evaluation, we demonstrate that the IQ-tree shows a performance that is the "best of both worlds" for a wide range of data distributions and dimensionalities.
Citation:
Stefan Berchtold, Christian Böhm, Hans-Peter Kriegel, Jörg Sander, H.V. Jagadish, "Independent Quantization: An Index Compression Technique for High-Dimensional Data Spaces," icde, pp.577, 16th International Conference on Data Engineering (ICDE'00), 2000
Usage of this product signifies your acceptance of the Terms of Use.