Parallel Techniques for Big Data Analytics

TL;DR: The experimental results show that by using the parallel technique, the indexing and mining of very large volumes of time series can now be done in very small execution times, which are impossible to achieve using traditional centralized approaches.

Abstract: Nowadays, we are witnessing the production of large volumes of data in many applications domains like social networks, medical monitoring, weather forecasting, biology, agronomy, earth monitoring, etc. Analyzing this data would help us to extract a lot of hidden knowledge about the events happened or to be happened in the future. However, traditional data analytics techniques are not efficient for analyzing such data volumes. A promising solution for improving the performance of data analytics is to take advantage of the computing power of distributed systems and parallel frameworks such as Spark. In this HDR manuscript, I describe my research activities for developing parallel and distributed techniques to deal with two main data analytics problems: 1) similarity search over time series; 2) maximally informative k-itemsets mining. The first problem, i.e., similarity search over time series, is very important for many applications such as fraud detection in finance, earthquake prediction, plant monitoring, etc. In order to improve the performance of similarity queries, index construction is one of the most popular techniques, which has been successfully used in a variety of settings and applications. In our research activities, we took advantage of parallel and distributed frameworks such as Spark, and developed efficient solutions for parallel construction of tree-based and grid-based indexes over large time series datasets. We also developed efficient algorithms for parallel similarity search over distributed time series datasets using indexes. The second problem, i.e., maximally informative k-itemsets mining (miki for short), is one of the fundamental building bricks for exploring informative patterns in databases. Efficient miki mining has a high impact on various tasks such as supervised learning, unsupervised learning or information retrieval, to cite a few. A typical application is the discovery of discriminative sets of features, based on joint entropy, which allows distinguishing between different categories of objects. Indeed, with massive amounts of data, the miki mining is very challenging, due to high number of entropy computations. An efficient miki mining solution should scale up with the increase in the size of the itemsets, calling for cutting edge parallel algorithms and high performance computation of miki. We developed such a parallel solution that makes the discovery of miki from a very large database (up to Terabytes of data) simple and effective.