An Optimization Method for Multiple Persistency Requirements on Stream Management System

TL;DR: The main idea of the optimization is to reduce the writing costs of DBMSs by sharing common store-operators and creates feasible processing plans for multiple persistency requirements.

Abstract: Today, the amount of data delivered as data streams has been increasing, and requirements over data streams have a great variety. We have been developing a stream management system called Harmonica by combining a stream processing engine and DBMSs to fulfill such requirements. Generally, throughputs of DBMSs are slower than those of stream processing engines with a high-speed memory. It is very difficult to process a huge number of continuous queries which request the system to store data into DBMSs. In order to overcome this bottleneck, we propose an optimization method for multiple persistency requirements. The main idea of our optimization is to reduce the writing costs of DBMSs by sharing common store-operators. Our optimization method creates feasible processing plans. In this paper, we present the algorithm of our method and the results of experiment by using Harmonica System.