loading...
Enabling Real-Time Querying of Live and Historical Stream Data
Banff, Alberta, Canada July 09-July 11
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/SSDBM.2007.3419th International Conference on Scie ...
 This Article 
 
PDF
HTML
 
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Frederick Reiss, Lawrence Berkeley National Laboratory, USA; University of California, Berkeley, USA; IBM Almaden Research Center, USA
Kurt Stockinger, Lawrence Berkeley National Laboratory, USA
Kesheng Wu, Lawrence Berkeley National Laboratory, USA
Arie Shoshani, Lawrence Berkeley National Laboratory, USA
Joseph M. Hellerstein, University of California, Berkeley, USA
Applications that query data streams in order to identify trends, patterns, or anomalies can often benefit from comparing the live stream data with archived historical stream data. However, searching this historical data in real time has been considered so far to be prohibitively expensive. One of the main bottlenecks is the update costs of the indices over the archived data. In this paper, we address this problem by using our highly-efficient bitmap indexing technology (called FastBit) and demonstrate that the index update operations are sufficiently efficient for this bottleneck to be removed. We describe our prototype system based on the TelegraphCQ streaming query processor and the FastBit bitmap index. We present a detailed performance evaluation of our system using a complex query workload for analyzing real network traffic data. The combined system uses TelegraphCQ to analyze streams of traffic information and FastBit to correlate current behaviors with historical trends. We demonstrate that our system can simultaneously analyze (1) live streams with high data rates and (2) a large repository of historical stream data.
Citation:
Frederick Reiss, Kurt Stockinger, Kesheng Wu, Arie Shoshani, Joseph M. Hellerstein, "Enabling Real-Time Querying of Live and Historical Stream Data," ssdbm, pp.28, 19th International Conference on Scientific and Statistical Database Management (SSDBM 2007), 2007
Usage of this product signifies your acceptance of the Terms of Use.


Suggestions