Database patchwork on the Internet

TL;DR: A distributed database system to be used in the Internet capable of dynamically loading (external) query operators, written in Java and supplied by any function provider, and executing these query operators in concert with pre-defined and other external operators in order to evaluate a query.

Abstract: Naturally, data processing requires three kinds of resources: the data itself,the functionality (i.e. database operations) andthe machines on which to run the operations.Because of the Internet we believe that in the long run there will be alternative providers for all of these three resources for any given application. Data providers will bring more and more data and more and more different kinds of data to the net. Likewise, function providers will develop new methods to process and work with the data; e.g., function providers might develop new algorithms to compress data or to produce thumbnails out of large images and try to sell these on the Internet. It is also conceivable, that some people allow other people to use spare cycles of their idle machines in the Internet (as in the Condor system of the University of Wisconsin) or that some companies (cycle providers) even specialize on selling computing time to businesses that occasionally need to carry out very complex operations for which regular hardware is not sufficient.At the University of Passau, we are currently developing a distributed database system to be used in the Internet. The goal is to ultimately have a system which is able to run on any machine, manage any kind of data, import any kind of data from other systems and import any kind of database operations. The system is entirely written in Java. One of the most important features of the system is that it is capable of dynamically loading (external) query operators, written in Java and supplied by any function provider, and executing these query operators in concert with pre-defined and other external operators in order to evaluate a query. Compared to object-relational database systems, which allow to integrate external data and functionality by the means of extensions (datablades, extenders or cartridges) or heterogeneous database systems such as Garlic [MS97] or Tsimmis [GMPQ+97], our approach makes it possible to place external query operators anywhere in a query evaluation plan as opposed to restricting the placement of external operations to the “access level” of plans. It would, for example, be possible to make our system execute a completely new relational join method, if somebody finds a new join method which is worth-while implementing. Because our system is written in Java, it is highly portable and could be used by data, function and cycle providers with almost no effort. Furthermore, our query engine is, of course, completely distributed providing all the required infrastructure for server-server communication, name services, etc.