1. Introduction and Overview. 2. Detecting Influential Observations and Outliers. 3. Detecting and Assessing Collinearity. 4. Applications and Remedies. 5. Research Issues and Directions for Extensions. Bibliography. Author Index. Subject Index.

Regression Diagnostics: Identifying Influential Data and Sources of Collinearity

■ Data mining and knowledge discovery in databases have been attracting a significant amount of research, industry, and media attention of late. What is all the excitement about? This article provides an overview of this emerging field, clarifying how data mining and knowledge discovery in databases are related both to each other and to related fields, such as machine learning, statistics, and databases. The article mentions particular real-world applications, specific data-mining techniques, challenges involved in real-world applications of knowledge discovery, and current and future research directions in the field.

/pdf/from-data-mining-to-knowledge-discovery-in-databases-4fznxuujb9.pdf

From Data Mining to Knowledge Discovery in Databases

On robust estimation of the location parameter

This chapter contains sections titled: Artificial Neural Networks, Neural Network Learning Algorithms, What a Perceptron Can and Cannot Do, Connectionist Models in Cognitive Science, Neural Networks as a Paradigm for Parallel Processing, Hierarchical Representations in Multiple Layers, Deep Learning

/pdf/neural-networks-and-deep-learning-3ccs3ain8p.pdf

Neural Networks and Deep Learning

Survey of previous comparisons and theoretical work descriptions of methods dataset descriptions criteria for comparison and methodology (including validation) empirical results machine learning on machine learning.

Machine Learning: Neural and Statistical Classification

Ensemble methods have been called the most influential development in Data Mining and Machine Learning in the past decade. They combine multiple models into one usually more accurate than the best of its components. Ensembles can provide a critical boost to industrial challenges -- from investment timing to drug discovery, and fraud detection to recommendation systems -- where predictive accuracy is more vital than model interpretability. Ensembles are useful with all modeling algorithms, but this book focuses on decision trees to explain them most clearly. After describing trees and their strengths and weaknesses, the authors provide an overview of regularization -- today understood to be a key reason for the superior performance of modern ensembling algorithms. The book continues with a clear description of two recent developments: Importance Sampling (IS) and Rule Ensembles (RE). IS reveals classic ensemble methods -- bagging, random forests, and boosting -- to be special cases of a single algorithm, thereby showing how to improve their accuracy and speed. REs are linear rule models derived from decision tree ensembles. They are the most interpretable version of ensembles, which is essential to applications such as credit scoring and fault diagnosis. Lastly, the authors explain the paradox of how ensembles achieve greater accuracy on new data despite their (apparently much greater) complexity.This book is aimed at novice and advanced analytic researchers and practitioners -- especially in Engineering, Statistics, and Computer Science. Those with little exposure to ensembles will learn why and how to employ this breakthrough method, and advanced practitioners will gain insight into building even more powerful models. Throughout, snippets of code in R are provided to illustrate the algorithms described and to encourage the reader to try the techniques. (edited by author)

Ensemble Methods in Data Mining: Improving Accuracy Through Combining Predictions

The quest to find models usefully characterizing data is a process central to the scientific method and has been carried out on many fronts. Researchers from an expanding number of fields have designed algorithms to discover rules or equations that capture key relationships between variables in a database. Some modern heuristic modeling approaches seem to have gained in popularity partly as a way to "avoid statistics" while still addressing challenging induction tasks. Yet, there are useful distinctives in what may be called a "statistical viewpoint", and we review here some major advances in statistics from recent decades that are applicable to Knowledge Discovery in Databases.

/pdf/a-statistical-perspective-on-kdd-gn8sm0qdv1.pdf

A statistical perspective on KDD

It is our great pleasure to welcome you to the Fifteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'09). This year is special in that it marks the first time that KDD is held in Europe, and where better to host it than beautiful Paris! As the flagship conference in the field, KDD continues to provide a highly competitive forum for reporting the latest and the best developments in the research and application of data mining and knowledge discovery worldwide.

KDD'09 received a record number of 659 submissions, more than 10% up from last year. The Research Track received 537 submissions and the Industrial and Government Applications Track received 122 submissions (an increase of nearly half over last year). For the Research Track, the program committee accepted 105 papers, 50 of which (9.3% of the submitted papers) were chosen for a 25 minute oral presentation and the remaining 55 (10.2%) for a 15 minute presentation. The corresponding numbers for the Industrial Track are 12 (9.8%) and 22 (18.0%). All accepted papers are given up to 9 pages in the proceedings, and all accepted papers are also given poster presentation opportunities in one of the two evening poster sessions during the conference.

In addition to the paper presentations, the conference features nine tutorials, eleven workshops, one panel, the hugely successful KDD-Cup competition, a demo session, an exhibition, evening industry sessions (new!), and three invited talks by David Hand (Imperial College London, UK), Heikki Mannila (University of Helsinki, Finland) and Stanley Wasserman (Indiana University, USA). The Industrial Track includes two additional invited presentations by Ravi Kumar (Yahoo! Research, USA) and Ashok Srivastava (NASA Ames, USA).

Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining

Inductive modeling or “machine learning” algorithms are able to discover structure in high-dimensional data in a nearly automated fashion. These adaptive statistical methods — including decision trees, polynomial networks, projection pursuit models, and additive networks — repeatedly search for, and add on, the model component judged best at that state. Because of the huge model space of possible components, the choice is typically greedy; that is, optimal only in the very short term. In fact, it is usual for the analyst and algorithm to be greedy at three levels: when choosing a 1) term within a model, 2) model within a family, and 3) family within a wide collection of methods. It is better, we argue, to “take a longer view” in each stage. For the first stage (term selection) examples are presented for classification using decision trees and estimation using regression. To improve the third stage (method selection) we propose fusing information from disparate models to make a combined model more robust. (Fused models merge their output estimates but also share information on, for example, variables to employ and cases to ignore.) Benefits of fusing are demonstrated on a challenging classification dataset, where the task is to infer the species of a bat from its chirps.

Heuristic Search for Model Structure: the Benefits of Restraining Greed

Structurally adaptive methods -from decision trees and polynomial networks, to projection pursuit models, additive networks, and cascade correlation neural networks -iteratively add components in an attempt to construct models with complexity appropriate to the data. The diverse basis functions and search strategies employed usually lead to a distribution of results (with rankings hard to predict priori) yet, robustly combining the output estimates can achieve a consensus model with properties often superior to the best of the individual models. Additionally, other information learned by some of the modeling techniques can be shared to the benefit of the fused system, including identification of key variables to employ and outlying cases to ignore. This paper describes the fused model developed for a small but challenging classification dataset (where one infers the species of a bat from its chirps) and introduces a robust method of combining the outputs of diverse models. Inductive Modeling Toolbox Though new inductive methods are introduced continually, they appear to be motivated by only a handful of key underlying ideas, and it is likely that a suite of methods can be identified which will essentially span "method space". In a readable empirical survey, Michie, Spiegelhalter and Taylor (1994)t suggested such a sufficient toolbox would include linear discrimination, decision trees, and k-nearest neighbors, as well as the more recent methods of projection pursuit and radial basis functions. Elder and Pregibon (1996) further suggest adding polynomial networks and (perhaps) adaptive splines and neural networks to the set covering methods, due to some relatively unique properties of those algorithms. Four of these methods -decision trees, k-nearest neighbors, neural networks, and polynomial networks -were selected to be used, in a multi-stage procedure, to address a chaUenging classification problem. Reviewed by Elder (1996). Application Example: Identifying Bat Species Tracking populations of potentially endangered bats can be simplified if similar species can be distinguished using features of their sophisticated echolocation signals. Data recently collected and analyzed by Kaefer et al. (1996) demonstrated that the necessary information is present in the signals, but that a fused model (described here) seems required to extract it. The data was obtained by capturing and labeling bats, recording several of their in-flight biosonar calls, extracting Fourier and time-frequency features from such "chirps", and iterating the signal feature extraction phase after analyzing early returns from individual induction algorithms charged with constructing discriminating classifiers. The data consists of 93 cases representing 18 different bats (with 3-8 signals per individual) from 5 species (some quite related physically) 2 all of which emit calls in the FM range (a spectrum area previously unexplored for the purposes of discrimination). Because there were multiple signals from each individual, a voting mechanism could be employed to arrive at a final classification for each set (bat). That is, bat was counted as correctly classified if a plurality of its acceptable signals were correctly identified. (The acceptance process is defined below.) For each algorithm, cross-validation accuracy was measured for both case-wise and bat-wise identification. That is, each method trained a model using 17 of the bat signal sets, then tested it on the 18th set (with this repeated 18 times so each set could be the hold-out sample in turn) and the collection of test results accumulated. Thus, the results reflect the accuracy of each algorithm (represented by a bundle of models) when applied to new individuals, as is required to estimate potential field performance.

/pdf/fusing-the-results-of-diverse-algorithms-2kfjbem6bh.pdf

Fusing the Results of Diverse Algorithms

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

John Elder

Author Tools

Chat about Author