Parallel processing of continuous data streams

TL;DR: The data stream processing as the processing program read continuously the data stream records and after some postponement time it produces the answer also continuously, and the process produces correct answer without loss of data.

Abstract: The continuous data streams produced by sensor applications wil have higher and higher importance in the practice in the future. Several applications generate data streams and the processing of these data streams needs high computing capacity. One of the suitable ways to provide the high capacity is the parallel computing. The processing of continuous data streams using parallel systems raises several interesting, important questions i.e. time postponements, feedbacks, suitable distribution. I INTRODUCTION Sensor applications are frequently used in the practice for indicating and/or signaling of the actual state of observed processes. More and more complex solutions are needed to use more than one data streams, historical data from databases or files and higher and higher computing capacities. One of the cheapest ways to provide the high computing capacity is the parallel processing. The parallel processing of continuous data streams raises some questions. We can consider the data stream processing as the processing program read continuously the data stream records and after some postponement time it produces the answer also continuously. The output of the data stream processing program is considered also data stream. (Fig. 1) Figure 1 Simple data stream processing program The notation stra,b symbolizes the data stream produced by "prog. a.", processed by "prog. b.". The general scheme of the data stream processing program is shown in Fig. 2. Generally the program processes one or more input streams, one uses historical (permanent) or active (changing data content) databases and/or files, and one produces one or more output data streams. Let vsiN denote the speed of striN data stream. The unit of the speed is considered as the number of the appearing records in data stream during one second. Let VPiN denote the speed of the processing, i.e. the number of the records processed by the N-th program arriving through stri,N data stream. The suitable cases are: VSi,N < VPiN and vsi,N-vpi,. The continuous existence of the latter case is not prospective. clb dlb Figure 2 General scheme of the data stream processing program Moreover the VPi,N proceeding speed would be decreased by increasing of the databases. We must count on cases described by relations vsiN > vpj,N. When vsiN > vpiN occurs for a short time, the usage of buffers would be suitable. This fact causes the increase of time-delay of the process but the process produces correct answer without loss of data. When the VSiN > VP4N …