Towards Optimal Multi-Dimensional Query Processing with Bitmap Indices

- 30 Sep 2005

TL;DR: The authors' experimental evaluation with real life data shows an average I/O cost improvement of at least a factor of 10 for multi-dimensional queries on datasets from two different applications and indicates that the speedup increases with the number of query dimensions.

Abstract: Bitmap indices have been widely used in scientific applications and commercial systems for processing complex, multi-dimensional queries where traditional tree-based indices would not work efficiently. This paper studies strategies for minimizing the access costs for processing multi-dimensional queries using bitmap indices with binning. Innovative features of our algorithm include (a) optimally placing the bin boundaries and (b) dynamically reordering the evaluation of the query terms. In addition, we derive several analytical results concerning optimal bin allocation for a probabilistic query model. Our experimental evaluation with real life data shows an average I/O cost improvement of at least a factor of 10 for multi-dimensional queries on datasets from two different applications. Our experiments also indicate that the speedup increases with the number of query dimensions.

AI Agents for this Paper

Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps

Most frequently asked questions

1. What contributions have the authors mentioned in the paper "Towards optimal multi-dimensional query processing with bitmap indices" ?

This paper studies strategies for minimizing the access costs for processing multi-dimensional queries using bitmap indices with binning.

2. What have the authors stated for future works in "Towards optimal multi-dimensional query processing with bitmap indices" ?

Future work involves testing their optimization techniques against other binning strategies.. A further direction of future work is to design bin allocation algorithms for probabilistic queries with attribute dependencies.

Fig. 7. Distribution of attributes (a) x-velocity and (b)density.

Fig. 1. Two-sided range query 8 < A < 37 on a bitmap index with binning.

Fig. 2. Optimal bin boundaries for various data and query distributions. The horizontal lines represent range queries, e.g. 350 ≤ A < 1201. The vertical lines indicate the optimal bin boundaries that are calculated using our dynamic programming algorithm taking into account both the data and the query distribution.

Fig. 3. Optimal bin allocation as a function of candidate selectivity.

Table 2. Probabilities for attributes to be contained in a query expression along with the respective optimal number of bins.

Fig. 6. Cost improvement factor of Opt-binning over equi-depth binning for multidimensional queries. For 4-dimensional queries, the cost improvement of Opt-binning over equi-depth binning is about a factor of 9.

Citations

•10.5170/CERN-2005-002.1131

Grid Collector: Using an event catalog to speed up user analysisin distributed environment

Kesheng Wu, +4 more

- 01 Nov 2004

TL;DR: For a user that typically performs computation on 50percent of the events, using Grid Collector could reduce the turn aroundtime by 30 percent and significantly improve user productivity.

...read moreread less

•Journal Article

Optimizing I/O costs of multi-dimensional queries using bitmap indices

Doron Rotem, +2 more

- 01 Jan 2005

- Lecture Notes in Computer Science

TL;DR: An efficient candidate check algorithm based on attribute value distribution, query distribution as well as query selectivity with respect to each dimension is presented and it is shown that re-ordering the dimensions during query evaluation can be used to reduce I/O costs.

...read moreread less

•Journal Article•10.1016/J.JCSS.2012.09.011

Optimizing adaptive multi-route query processing via time-partitioned indices

Karen Works, +2 more

- 01 May 2013

- Journal of Computer and System Sciences

TL;DR: The proposed Adaptive Multi-Route Index (AMRI) employs a bitmap time-partitioned design that serves a diverse ever changing workload of query access patterns and remains lightweight in terms of maintenance and storage requirements.

...read moreread less

Proceedings Article•10.1109/IPDPSW.2010.5470845

Index tuning for adaptive multi-route data stream systems

Karen Works, +2 more

- 19 Apr 2010

TL;DR: This experimental study using synthetic data streams has demonstrated that AMRI strikes a balance between supporting effective query processing in dynamic stream environments while keeping the overhead to a minimum.

...read moreread less

Time-partitioned Index Design for Adaptive Multi-Route Data Stream Systems utilizing Heavy Hitter Algorithms

Karen Works, +2 more

- 01 Jan 2010

TL;DR: This proposed Adaptive Multi-Route Index (AMRI) employs a bitmap time-partitioned design that while being versatile in serving a diverse ever changing workload of multiple query access patterns remains lightweight in terms of maintenance and storage requirements.

...read moreread less

References

Journal Article•10.1145/248603.248616

An overview of data warehousing and OLAP technology

Surajit Chaudhuri, +1 more

- 01 Mar 1997

TL;DR: An overview of data warehousing and OLAP technologies, with an emphasis on their new requirements, is provided, based on a tutorial presented at the VLDB Conference, 1996.

...read moreread less

•Proceedings Article•10.1145/253260.253268

Improved query performance with variant indexes

Patrick O'Neil, +1 more

- 01 Jun 1997

TL;DR: A new method whereby multi-dimensional group-by queries, reminiscent of OLAP/Datacube queries but with more flexibility, can be very efficiently performed is introduced.

...read moreread less

601

Proceedings Article•10.1145/276304.276336

Bitmap index design and evaluation

Chee-Yong Chan, +1 more

- 01 Jun 1998

TL;DR: A general framework to study the design space of bitmap indexes for selection queries and examine the disk-space and time characteristics that the various alternative index choices offer, and describes a bitmap-index-based evaluation algorithm that represents an improvement over earlier proposals.

...read moreread less

441

Book Chapter•10.1007/3-540-51085-0_42

Model 204 Architecture and Performance

Patrick O'Neil

- 28 Sep 1987

TL;DR: Some of the major architectural principles of MODEL 204, a commercial database product of Computer Corporation of America for the IBM 370 computer, are presented and how these principles result in exceptional performance advantages performing certain common tasks in complex query processing are indicated.

...read moreread less

236

Proceedings Article•10.1109/ICDE.1998.655780

Encoded bitmap indexing for data warehouses

Ming-Chuan Wu, +1 more

- 23 Feb 1998

TL;DR: An encoded bitmap indexing for DWs is proposed which improves the performance of known bit map indexing in the case of large cardinality domains and is compared with related techniques, such as bit slicing, projection-, dynamic-, and range-based indexing.

...read moreread less

200

...

Expand

Towards Optimal Multi-Dimensional Query Processing with Bitmap Indices

Chat with Paper

AI Agents for this Paper

Most frequently asked questions

1. What contributions have the authors mentioned in the paper "Towards optimal multi-dimensional query processing with bitmap indices" ?

2. What have the authors stated for future works in "Towards optimal multi-dimensional query processing with bitmap indices" ?

Figures

Citations

Grid Collector: Using an event catalog to speed up user analysisin distributed environment

Optimizing I/O costs of multi-dimensional queries using bitmap indices

Optimizing adaptive multi-route query processing via time-partitioned indices

Index tuning for adaptive multi-route data stream systems

Time-partitioned Index Design for Adaptive Multi-Route Data Stream Systems utilizing Heavy Hitter Algorithms

References

An overview of data warehousing and OLAP technology

Improved query performance with variant indexes

Bitmap index design and evaluation

Model 204 Architecture and Performance

Encoded bitmap indexing for data warehouses

Related Papers (5)

Improving the Performance of High-Energy Physics Analysis through Bitmap Indices

A General Analytical Model for Spatial and Temporal Performance of Bitmap Index Compression Algorithms in Big Data

Performance Evaluation and Optimization of Multi-Dimensional Indexes in Hive

FastBit: an efficient indexing technology for accelerating data-intensive science

Big data-based online analytical processing system and method