Abstract: The aim of the Stat Log project is to compare the performance of statistical, machine learning, and neural network algorithms, on large real world problems. This paper describes the completed work on classification in the StatLog project. Classification is here defined to be the problem, given a set of multivariate data with assigned classes, of estimating the probability from a set of attributes describing a new example sampled from the same source that it has a pre-defined class. We gathered together a representative collection of algorithms from statistics (Naive Bayes, K-nearest Neighbour, Kernel density, Linear discriminant, Quadratic discriminant, Logistic regression, Projection pursuit, Bayesian networks), machine learning (CART, C4.5, NewID, AC2, CAL5, CN2, ITrule — only propositional symbolic algorithms were considered), and neural networks (Backpropagation, Radial basis functions, Kohonen). We then applied these algorithms to eight large real world classification problems: four

Abstract: In this chapter, we present a polynomial time algorithm, called a k-minimal multiple generalization (k-mmg) algorithm, where k 1, and its application to inductive learning problems. The algorithm is a natural extension of the least general generalization algorithm developed by Plotkin and Reynolds. Given a nite set of ground rst-order terms, the k-mmg algorithm generalizes the examples by at most k rst-order terms, while Plotkin's algorithm does by a single rst-order term. We apply the k-mmg algorithm to several learning problems in inductive logic programming, and knowledge discovery in databases.

Abstract: The BOXES algorithm of Michie and Chambers (1968) has proved to be an effective and flexible method for learning to control dynamic systems. The algorithm, in its original form has been used a benchmark for many experiments in control tasks such as pole balancing. Recent work in our laboratory has shown that the BOXES algorithm can be improved to yield very good learning rates. We describe experiments on a variety of update functions and discuss their robustness. We also develop the notion of freezing of BOXES, suggested by Michie and implemented by Bain (1990). We have also been concerned with synthesising a readable account of the control strategy employed by a set of boxes. Some preliminary work has begun in combining decision tree learning algorithms with BOXES. Using this method, we regard BOXES as the acquirer of sub-cognitive skills and the decision tree induction as a means of introspecting on the learned strategy to generate understandable control rules.

Abstract: Non-demonstrative or inductive reasoning is a crucial component in the skills of a learner. A leading candidate for this form of reasoning involves the automatic formation of hypotheses. Initial successes in the construction of propositional theories have now been followed by algorithms that attempt to generalise sentences in the predicate calculus. An important defect in these new-generation systems is the lack of a clear model for theory justiication. In this paper we describe a method of evaluating the signiicance of a hypothesis based on the degree to which it allows compression of the observed data with respect to prior knowledge. This can be measured by comparing the lengths of the input and output tapes of a reference Turing machine which will generate the examples from the hypothesis and a set of derivational proofs. The model extends an earlier approach of Muggleton by allowing for noise. The truth values of noisy instances are switched by making use of correction codes. The utility of compression as a signiicance measure is evaluated empirically in three independent domains. In particular, the results show that the existence of compression distinguishes a larger number of signiicant clauses than other signiicance tests. The method also appears to distinguish noise as incompressible data.

Abstract: Turing's best known work is concerned with whether universal machines can decide the truth value of arbitrary logic formulae. However, in this paper it is shown that there is a direct evolution in Turing's ideas from his earlier investigations of computability to his later interests in machine intelligence and machine learning. Turing realised that machines which could learn would be able to avoid some of the consequences of Godes and his results on incompleteness and undecidability. Machines which learned could continuously add new axioms to their repertoire. Inspired by a radio talk given by Turing in 1951, Christopher Strachey went on to implement the world's first machine learning program. This particular first is usually attributed to A.L. Samuel. Strachey's program, which did rote learning in the game of Nim, preceded Samuel's checker playing program by four years. Neither Strachey's nor Samuel's system took up Turing's suggestion of learning logical formulae. Developments in this area were delayed until Gordon Plotkin's work in the early 1970's. Computer-based learning of logical formulae is the central theme of the research area of Inductive Logic Programming, which grew directly out of the earlier work of Plotkin and Shapiro. In the present paper the author describes the state of this new field and discusses areas for future development.

Abstract: This paper presents a speed-up learning method for logic programs, which accelerates a program by composing macro clauses based on partial structures of proof trees. Many systems have been proposed for composing useful macros, e.g., some of them select macros that connects two peaks in a heuristic function. Another employs heuristics that select useful macros. Although they work well in some domains, such methods depend on domain-dependent heuristics that have to be exploited by their users. We propose a heuristic-independent mechanism by detecting backtrack-ing. The method uses a dead-end path as a negative explanation tree, compares it with positive one, and nds a rst dierent node to remove its corresponding rule by composing a macro. Repeated substructures in such a macro are then combined by applying the generalize-number technique and by sharing common substructures. Experimental results in STRIPS domain show that, by selecting an appropriate set of macros, 1) backtracking in solving training examples are suppressed, 2) its problem solving eciency does not deteriorate even after learning a number of examples, 3) after learning 30 training examples, no backtracking occurs in solving 100 test examples dierent from the training examples. In conclusion, the proposed method speeds up the problem solving from 10 to 100 times.

Abstract: This paper discusses large knowledge bases as software development tools which support the creativity of programming in the large. User requirements, architecture and internal knowledge representation language of large knowledge bases are considered. Higher order constraint networks are proposed for representing knowledge about computability. 1 Software reusability An important characteristic of the software development process is the degree of reusability of software. Simply speaking, knowledge once encoded in the form of programs must be reusable every time it would be needed in programming new problems. A natural way to reuse programs is to apply large software libraries. It is expected that this increases the productivity of software development and reliability of the software produced. However, with the exception of a small number of speciic applications, the software libraries of today tend to be very diicult to use. They lack comprehensive user interface, and require from the users too much eeorts of studying of documentation. One can use the following analogy. From a usability standpoint , software library is like an ordinary library of literature containing a large number of books, except that it has no comprehensive catalogue, the books dont have title pages, and they are stored in a random order and are accessible only by numbers which are their formal addresses. Attempts are being made to build knowledge bases which could provide a guidance in selecting suitable software from software libraries (Devanbu 1991). The goal of the present work is to propose a design for a knowledge base which would support automatic construction of large programs from their declarative speciications. Roughly speaking, we shall build a software library which contains two layers of knowledge, Figure 1.1. The lower layer is a repository of programs. These programs are not directly visible in the software development process. They are covered by the layer of knowledge about their applicability for solving diierent problems. This knowledge is visible to users (software developers) and it is represented in terms of concepts of a problem domains, not in terms of programs. Besides that, we distinguish between the internal knowledge 1

Abstract: Building Symbolic Representations of Intuitive Real-time Skills from Performance Data D. Michie and R. Camacho The Turing Institute, Glasgow, UK Abstract Real-time control skills are ordinarily tacit | their possessors cannot explicitly communicate them. But given su cient sampling of a trained expert's input{output behaviour, machine learning programs have been found capable of constructing rules which, when run as programs, deliver behaviours similar to those