Abstract: When high-dimensional data vectors are visualized on a two- or three-dimensional display, the goal is that two vectors close to each other in the multi-dimensional space should also be close to each other in the low-dimensional space. Traditionally, closeness is defined in terms of some standard geometric distance measure, such as the Euclidean distance, based on a more or less straightforward comparison between the contents of the data vectors. However, such distances do not generally reflect properly the properties of complex problem domains, where changing one bit in a vector may completely change the relevance of the vector. What is more, in real-world situations the similarity of two vectors is not a universal property: even if two vectors can be regarded as similar from one point of view, from another point of view they may appear quite dissimilar. In order to capture these requirements for building a pragmatic and flexible similarity measure, we propose a data visualization scheme where the similarity of two vectors is determined indirectly by using a formal model of the problem domain; in our case, a Bayesian network model. In this scheme, two vectors are considered similar if they lead to similar predictions, when given as input to a Bayesian network model. The scheme is supervised in the sense that different perspectives can be taken into account by using different predictive distributions, i.e., by changing what is to be predicted. In addition, the modeling framework can also be used for validating the rationality of the resulting visualization. This model-based visualization scheme has been implemented and tested on real-world domains with encouraging results.

Abstract: In this work we investigate some technical questions related to the application of neural networks in document classification. First, we discuss the effects of different averaging protocols for the \chi ^{2} statistic used to remove non-informative terms. This is an especially relevant issue for the neural network technique, which requires an aggressive dimensionality reduction to be feasible. Second, we estimate the importance of performance fluctuations due to inherent randomness in the training process of a neural network, a point not properly addressed in previous works. Finally, we compare the neural network results with those obtained using the best methods for this application. For this we optimize the network architecture by evaluating much larger nets than previously considered in similar studies in the literature.

Abstract: Regularities exist in datasets describing spatially distributed physical phenomena. Human experts often understand and verbalize the regularities as abstract spatial objects evolving coherently and interacting with each other in the domain space. We describe a novel computational approach for identifying and extracting these abstract spatial objects through the construction of a hierarchy of spatial relations. We demonstrate the approach with an application to finding pressure trough features in weather data sets.

Abstract: The discovery of characteristic rules is a well-known data mining task and has lead to several successful applications. However, because of the descriptive nature of characteristic rules, typically a (very) large number of them is discovered during the mining stage. This makes monitoring and control of these rules, in practice, extremely costly and difficult. Therefore, a selection of the most promising subset of rules is desirable. Some heuristic rule selection methods have been proposed in the literature that deal with this issue. In this paper, we propose an integer programming model to solve the problem of optimally selecting the most promising subset of characteristic rules. Moreover, the proposed technique enables to control a user-defined level of overall quality of the model in combination with a maximum reduction of the redundancy extant in the original ruleset. We use real-world data to empirically evaluate the benefits and performance of the proposed technique against the well-known RuleCover heuristic. Results demonstrate that the proposed integer programming techniques are able to significantly reduce the number of retained rules and the level of redundancy in the final ruleset. Moreover, the results demonstrate that the overall quality in terms of the discriminant power of the final ruleset slightly increases if integer programming methods are used.

Abstract: The problem of feature subset selection can be defined as the selection of a relevant subset of features which allows a learning algorithm to induce small high-accuracy models This problem is of primary important because irrelevant and redundant features may degrade the learner speed, especially in the context of high dimensionality, and reduce both the accuracy and comprehensibility of the induced model Two main approaches have been developed, the first one is algorithm-independent (filter approach) which considers only the data, when the second approach which is algorithm-dependent takes into account both the data and a given learning algorithm (wrapper approach) Recent work was developed to study the interest of the rough set theory and more particularly its notions of reducts and core to deal with the problem of feature subset selection Different methods were proposed to select features using both the core and the reduct concepts, whereas other researches show that useful feature subsets do not necessarily contain all features in cores In this paper, we underline the fact that rough set theory is concerned with deterministic analysis of attribute dependencies which are at the basis of the two notions of reduct and core We extend the notion of dependency which allows to find both deterministic and non-deterministic dependencies A new notion of strong reducts is then introduced and leads to the definition of strong feature subsets (SFS) The interest of SFS is illustrated by the improvement of the accuracy of C45 on real-world datasets Our study shows that generally the highest-accuracy-subset is not the best one as regards to the filter criteria The highest accuracy subset is found by the new approach with minimum cost The contribution of this work is four folds : (1) analysis of feature subset selection in the rough sets context, (2) introduction of new definitions based on a generalized rough set theory, ie, \alpha-RST, (3) reformulation of the selection problem, (4) description of a hybrid method combining combining both the filter and the wrapper approaches

Abstract: The goal of this research is the discovery of useful concepts in temporal medical databases. Building on previous experiments, we introduce TEMPADIS, the Temporal Pattern Discovery System, which uses an Event Set Sequence approach to discover sequential patterns in medical data. We discuss problems unique to mining medical databases and introduce techniques to overcome some of these problems. Verification results are presented based on a database of Human Immunodeficiency Virus (HIV) patients monitored over four years.

Abstract: Machine learning (ML) is a useful and productive component of data mining (DM). Given a large database, a learning algorithm induces a description of concepts (classes) which are immersed in a given problem area. The induction itself consists in searching usually a huge space of possible concept descriptions. There exist several paradigms for controlling this search. One of the promising and efficient paradigms are {\it genetic algorithms} (GAs). There have been done many research projects of incorporating genetic algorithms into the field of machine learning. This paper describes an efficient application of a GA in the attribute-based rule-inducing learning algorithm. Actually, a domain-independent GA has been integrated into the covering learning algorithm CN4, a large extension of the well-known algorithm CN2; the induction procedure of CN4 (beam search methodology) has been removed and the GA has been implanted into this shell. Genetic algorithms are capable of processing symbolic attributes in a simple, natural manner. The processing of numerical (continuous) attributes by genetic algorithms is not so straightforward. One feasible strategy is to discretize numerical attributes before a generic algorithm is called. There exist quite a few discretization preprocessors in data mining and machine learning. This paper describes a newer preprocessor for discretization (categorization) of numerical attributes. The genuine discretization procedures generate sharp bounds (thresholds) between intervals. It may result in capturing training objects from various classes (concepts) into one interval that will not be `pure'; this in particular happens near the interval borders. One feasible way how to eliminate such an impurity around the interval borders is to fuzzify them. The paper first introduces the methodology of our new learning algorithm, the genetic learner. Then the discretization/fuzzification preprocessor is presented. Finally, the paper compares the entire system (a preprocessor and genetic learner) with well-known covering as well as TDIDT learning algorithms.

Abstract: This paper presents a new approach to unsupervised pattern classification. The classification scheme consists of two main stages. The first one is an unsupervised fuzzy learning procedure, which al...

Abstract: Tree-structured classifiers have proved their ability to show good result in comparison with other classification techniques applied to real-world data which is usually noisy and uncertain. The purpose of this article is to survey a representative selection of existing types of tree-structured classifiers and evaluate their abilities to classify data sets with and without highly correlated attributes. The primary focus, however, is on identifying the suitability of applying tree-structured algorithms to data with interconnected attributes which is an essential feature of financial and business data. To carry out this study two financial data sets are used. The first data set contains quantitative data relating to a company's credit rating score. The second data set contains financial ratios related to company solvency. To determine the efficiency of different tree-structured algorithms five algorithms (four different types) were selected for comparison purposes. From the experimental results it is possible to see, that classification based on the mixed approach (NBTree) performed the best. Classifiers with a Bayesian approach also showed that they are stable.

Abstract: Most fuzzy controllers and fuzzy expert systems must predefine membership functions and fuzzy inference rules to map numeric data into fuzzy linguistic values and to make fuzzy reasoning work. Recently, fuzzy systems that automatically derive fuzzy if-then rules from numeric data have been developed. In this paper, we propose a new learning method to automatically derive membership functions and fuzzy if-then rules from a set of given training examples. This method adopts a different way in building initial membership functions, thus making the learning procedure simpler than that used in [10]. Experiments are also made to show the performance of the newly proposed learning algorithm.

Abstract: Feature selection is a central problem in data analysis that have received a significant amount of attention from several disciplines, such as machine learning or pattern recognition. However, most...

Abstract: During the last few years, fuzzy logic and Genetic algorithms (GAs) have been experiencing extremely rapid growth in the industrial world, where they have been shown to be very effective in solving real-world problems. In this paper, we present a coevolutionary inspired method, which combines sharing GA with a fuzzy clustering technique for multimodal function optimization. Without using any prior knowledge, this approach allows both location and maintenance of niches. Since the niche radii are continuously updated, a fine local tuning is also performed. Several well-known functions are used to test the performances of the proposed algorithm.

Abstract: WordNet, a hand-made, general-purpose, and machine-readable thesaurus, has been used in information retrieval research by many researchers, but failed to improve the performance of their retrieval system. Thereby in this paper we investigate why the use of WordNet has not been successful. Based on this analysis we propose a method of making WordNet more useful in information retrieval applications by combining it with other knowledge resources. A simple word sense disambiguation is performed to avoid misleading expansion terms. Experiments using several standard information retrieval test collections show that our method results in a significant improvement of information retrieval performance. Failure analysis were done on the cases in which the proposed method fail to improve the retrieval effectiveness. We found that queries containing negative statements and multiple aspects might cause problems in the proposed method and we also investigated the solution to these problems.

Abstract: Conventional algorithms for decision tree induction use an attribute-val ue representatio n scheme for instances. This paper explores the empirical consequences of using set-valued attributes. This simple representational extension, when used as a pre-processor for numeric data, is shown to yield significant gains in accuracy combined with attractive build times. It is also shown to improve the accuracy for the second best classification option, which has valuable ramifications for post-processing. To do so an intuitive and practical version of pre-pruning is employed. Moreover, the implementation of a simple pruning scheme serves as an example of pruning applicability over the resulted trees and also as an indication that the proposed discretization absorbs much of pruning potential. Finally, we construct several versions of the basic algorithm to examine the value of every component that comprises it.

Abstract: The trees constructed by decision tree induction programs are often unnecessarily large and containing substantial degrees of duplication because such programs typically build a separate subtree for each value of a categorical attribute. Attribute value grouping procedures attempt to avoid this problem by partitioning attribute values into groups, each of which gives rise to only one subtree. In this paper we raise and attempt to answer a number of questions about the performance of such procedures. We review the limited amount of research that has been done in this area and propose a number of novel attribute value grouping procedures. We then present the results of a systematic comparative study in which eight alternative attribute grouping procedures were evaluated using both artificial and real data sets. We found that attribute value grouping can produce substantial reductions in tree size and that the best methods produce average reductions approaching 50% for real data. We also found that there was no effect on the classification accuracy of the trees produced but the time required to produce them was reduced. The most surprising finding was that global methods, which group attribute values once prior to tree construction were superior to local methods, which repartition values throughout tree construction: they produce substantially smaller trees in less time that are marginally more accurate classifiers.

Abstract: Queries on probabilistic databases would be based on approximate matching rather than exact matching. This is partly due to the fact that the user may not know what are the exact probabilities of objects in a database. On the other hand, the domain of the attribute of a 1NF relational scheme is generally required finite. But the domain (0, 1] of the attribute that describes the probabilistic significance of an object is infinite. This means that it does not seem appropriate for approximate queries. In order to perform anything useful, a probabilistic data model is advocated for representing probabilistic data in this paper. The model is based on our definition of the nearest neighbor of data, which is used to measure the equality of probabilistic data. As a result, the approximation and infinite semantics of probabilistic data can be modeled in the nearest neighbor. Furthermore, a probabilistic relational algebra is also proposed so as to approximately query such databases.

Abstract: A fast procedure for classifying a given test pattern to one of its possible classes using both the K-NN decision rule and concepts of the fuzzy set theory is described in this paper. The method is divided into two steps; in the first step the K nearest neighbours are found using a fast procedure, whereas in the second step the test pattern is classified using a fuzzy distance measure. The fast K-NN algorithm proceeds in finding a region containing at least K neighbours around the test sample by utilising an ordered search procedure of the test point from three reference points. In the sequence the found region is modified in such a way that the real K nearest neighbours of the given point will be always inside it. Then a small number of distance calculations are required to identify the true K-nearest neighbours and the fuzzy measure is applied to classify the test pattern. The pre-processing load is quite moderate and computer simulation results show that the misclassification rate is lower than, or similar to, the crisp version, while presenting results richer in information content than its crisp counterpart. This rate is also kept low even in the case we do not perform modifications that ensure the true K-NN finding.

Abstract: Knowledge Discovery in Databases (KDD) has become a very attractive discipline both for research and industry within last few years. Its goal is to extract pieces of knowledge or `patterns' from us...

Abstract: Isolation and purification of the active principle within natural compounds plays an important role in drug development. MS (mass spectrometry) is used as a detector in HPLC (high performance liquid chromatography) systems to aid the determination of novel compound structures. Clustering techniques provide useful tools for intelligent data analysis within this context. In this paper, we analyse some representative clustering algorithms, describe the complexities of the mass spectrometry data generated by HPLC-MS systems, and provide a new algorithm for clustering, based on the needs of drug development. This new algorithm is based on the definition of a dynamic window within the instance space.

Abstract: The visualization of large text databases and document collections is an important step towards more flexible and interactive types of information access and retrieval. This paper presents a probab...

Abstract: This paper presents an implemented framework for intelligent system integration based on the concept of intercommunicating hybrids. The implemented toolset based on the framework is called the Intelligent Forecasters Construction Set (IFCS), which is a hybrid-programming environment that allows the developer to implement forecasters by means of neural network modules, object-oriented visual programming, knowledge-based programming and procedural programming. Neural network modules, rules, procedures and other intelligent techniques are encapsulated into blocks which can connect with each other as data flow diagrams for data processing. The flow diagrams can be organized into a hierarchy of workspaces to solve problems. The system was implemented on the real-time expert system shell G2^1, with G2 Diagnostic Assistant (GDA^1) and NeurOn-Line^1 (NOL) modules. The modularity of IFCS allows subsequent addition of other modules of intelligent techniques. The IFCS was used for developing forecasters of daily electricity demand and water demand at the City of Regina based on the idea of homogeneous multi-module system. In both cases, the data sets were separated into subclasses and each of them was modeled with a neural network module. The two problem domains were also modeled using a linear regression (LR) and a case based reasoning (CBR) program. The benefits of a multi-module neural network approach are discussed and some experimental results from the applications are presented.

Abstract: A novel similarity measure, proposed for clustering data with arbitrary distribution shapes, is developed. Such a new measure of similarity is employed in a dynamic model to collectively measure similarity among pattern vectors, which can help to achieve a more robust clustering performance than using the existing measures that are staticly and individually based on the distances among the isolated pairwise data. The experiment results demonstrated that the proposed neural network based on the new similarity measure has the capability to robustly and quickly cluster data on which Cluster-Detection-and-Labeling neural network fails.

Abstract: In hyperspectral image analysis, determining a distinct material number is an important task for subsequent classification processes. Identifying the number of distinct materials is essentially the same task as determining the intrinsic dimensionality of the imaging spectrometer data. Minimum noise fraction (MNF) transformation or noise-adjusted principal component analysis (NAPCA) is a highly effective means of determining the inherent dimensionality of image data. However, inaccuracy in the noise estimation degrades the validity of this estimation. To effectively resolve this problem, this work presents a Noise-Adjusted Transformed Gerschgorin Disk approach (NATGD) which incorporates the NAPCA method into a transformed Gerschgorin disk (TGD) approach. By noise-adjusted, Gerschgorin disks in NATGD can be formed into two distinct, non-overlapping collections; one for signals and the other for noises. Hence, the number of distinct materials can be visually determined by counting the number of Gerschgorin disks for signals. Experimental results demonstrate that the method proposed herein can effectively solve the intrinsic dimensionality problem.

Abstract: This paper describes a new approach to indexing time series. Indexing time series is more problematic than indexing text since in extreme we need to find all possible subsequences in the time series sequence. We propose to use signature files to index the time series and we also propose a new method to index the signature files to speed up the search of large time series. We propose a novel index structure, the signature tree, for time series indexing. We implemented the signature tree and we discuss its performance.

Abstract: An {\it Event History} consists of a series of events where each event has an associated time of occurrence; events mark the start or end of a {\it spell}. An episode is a sequence of spells, which is a subset of the sequence of all spells that an individual undergoes; the full set constitutes the Event History. In this paper we are concerned with rule discovery for such Event Histories. Frequently interest focuses not only on the episodes which comprise of sequences of spells but also on the spell durations. We have provided a methodology for discretising such durations which utilises phase-type distributions where the phases correspond to the discretised classes. We also discuss how the Dempster-Shafer Theory of Evidence may be utilised to provide a methodology for inducing rules from such data when they contains partial values. These are caused by spells which are incomplete, frequently because at the current time point there are spells still in progress. Our approach is illustrated using data concerning event histories for 6994 geriatric patients.

Abstract: Neural networks (NN) have made a great impact in modeling and synthesizing non-linear mapping of input-output space. In this paper, we describe the design and testing of a particular class of NN, radial basis function networks, for dryness prediction in a clothes dryer. Our objective is to design an improved, robust, accurate, and adaptive system for dryness prediction, leading to a low-cost, energy-efficient, electronically controlled clothes dryer. We synthesize a stepwise radial basis function network to predict clothes moisture content and optimize the number of required sensors, while providing learning capabilities to account for external disturbances. In addition, we quantize clothes moisture content into ``degrees of dryness'', thus enhancing the prediction accuracy. We show that this approach results in a predictor that is superior to non-linear regression and multi-layer perceptron tools.

Abstract: This paper presents a new approach to unsupervised pattern classification. The classification scheme consists of two main stages. The first one is an unsupervised fuzzy learning procedure, which allows, using a similarity measure and a corresponding threshold, to seek clusters within a set of totally unlabeled samples. It provides, for each detected cluster, a good initial prototype as well as the membership degree of each sample. The second stage is an optimization procedure involving the fuzzy c-means (FCM) algorithm. Both procedures are repeated for different values of the similarity threshold, and three validity criteria are used to assess and rank the quality of all resulting partitions. The effectiveness of this approach is demonstrated, for different parameter values, on both artificial and real test data.

Abstract: Knowledge Discovery in Databases (KDD) has become a very attractive discipline both for research and industry within last few years. Its goal is to extract pieces of knowledge or `patterns' from usually very large databases. It portrays a robust sequence of procedures or steps that have to be carried out to derive reasonable and understandable results. One of its components symbolizes an inductive process that induces the above pieces of knowledge; usually it is Machine Learning (ML). However, most of the machine learning algorithms require perfect data in a reasonable format. Therefore, some preprocessing routines as well as postprocessing ones should fill the entire chain of data processing. This paper overviews and discusses the knowledge discovery process and its methodology as a series of several steps which include machine learning, preprocessing of data, and postprocessing of the results induced.