Data Mining - Concepts and Techniques.

Anomaly detection is an important problem that has been researched within diverse research areas and application domains. Many anomaly detection techniques have been specifically developed for certain application domains, while others are more generic. This survey tries to provide a structured and comprehensive overview of the research on anomaly detection. We have grouped existing techniques into different categories based on the underlying approach adopted by each technique. For each category we have identified key assumptions, which are used by the techniques to differentiate between normal and anomalous behavior. When applying a given technique to a particular domain, these assumptions can be used as guidelines to assess the effectiveness of the technique in that domain. For each category, we provide a basic anomaly detection technique, and then show how the different existing techniques in that category are variants of the basic technique. This template provides an easier and more succinct understanding of the techniques belonging to each category. Further, for each category, we identify the advantages and disadvantages of the techniques in that category. We also provide a discussion on the computational complexity of the techniques since it is an important issue in real application domains. We hope that this survey will provide a better understanding of the different directions in which research has been done on this topic, and how techniques developed in one area can be applied in domains for which they were not intended to begin with.

/pdf/anomaly-detection-a-survey-1x33g9m0a3.pdf

Anomaly detection: A survey

Matrix Factorization Techniques for Recommender Systems

Journal of Statistical Mechanics: theory and experiment

The article reports a field study which examined the mental models of 80 undergraduates seeking information for either a history or psychology course essay when they were in an early, exploration stage of researching their essay. This group is presently at a disadvantage when using thesaurus-type schemes in indexes and online search engines because there is a disconnect between how domain novice users of IR systems represent a topic space and how this space is represented in the standard IR system thesaurus. The study attempted to (a) ascertain the coding language used by the 80 undergraduates in the study to mentally represent their topic and then (b) align the mental models with the hierarchical structure found in many thesauri. The intervention focused the undergraduates' thinking about their topic from a topic statement to a thesis statement. The undergraduates were asked to produce three mental model diagrams for their real-life course essay at the beginning, middle, and end of the interview, for a total of 240 mental model diagrams, from which we created a 12-category mental model classification scheme. Findings indicate that at the end of the intervention, (a) the percentage of vertical mental models increased from 24 to 35p of all mental models; but that (b) 3rd-year students had fewer vertical mental models than did 1st-year undergraduates in the study, which is counterintuitive. The results indicate that there is justification for pursuing our research based on the hypothesis that rotating a domain novice's mental model into a vertical position would make it easier for him or her to cognitively connect with the thesaurus's hierarchical representation of the topic area. © 2007 Wiley Periodicals, Inc.

A classification of mental models of undergraduates seeking information for a course essay in history and psychology: Preliminary investigations into aligning their mental models with online thesauri

We present XLearner, a novel tool that helps the rapid development of XML mapping queries written in XQuery. XLearner is novel in that it learns XQuery queries consistent with given examples (fragments) of intended query results. XLearner combines known learning techniques, incorporates mechanisms to cope with issues specific to the XQuery learning context, and provides a systematic way for the semiautomatic development of queries. We describe the XLearner system. It presents algorithms for learning various classes of XQuery, shows that a minor extension gives the system a practical expressive power, and reports experimental results to demonstrate how XLearner outputs reasonably complicated queries with only a small number of interactions with the user.

/pdf/a-machine-learning-approach-to-rapid-development-of-xml-38mdaoyilj.pdf

A machine learning approach to rapid development of XML mapping queries

Recently, with the rapid spread of XML format, it has become popular that large-scale data, whose size range from several hundreds of MB to several GB, are described by XML. For the purpose of providing fast and reliable means for storage and retrieval of huge XML data, it is a reasonable choice for us to use XML databases. In fact, there are many ways to realize XML databases, but relational XML database, in that an XML data is mapped to relational tables and query processing is enabled in terms of SQL queries, is one of the most popular way to implement XML databases. However, some researchers have pointed out that the performance of relational XML databases degrades when dealing with such huge XML data. In this study, we propose a scheme for parallel processing of XML data using PC Clusters. First, we discuss how to decompose XML data so that we can perform parallel processing of XML queries. We give the definitions of vertical and horizontal decomposition of XML data based on decomposition of schema graph and XML instances, respectively. To allocate decomposed XML data to cluster nodes, we give an algorithm for computing pseudo-optimal assignment of XML fragments like greedy method in the light of XML query workload. Finally, we experimentally evaluate the effectiveness of the proposed method.

Processing XPath Queries in PC-Clusters Using XML Data Partitioning

Outlier detection over data streams is an increasingly important task in data mining. Traditional distance-based data stream outlier detection is unsuitable for high-dimensional data sets, since the discrimination of distances between different data points becomes rather poor in high dimensional space. ABOD (Angle-based Outlier Detection) is an effective approach to detecting outliers in high-dimensional space. In this paper, the problem of continuous ABOD over data streams is studied. Generally, only a few data objects may change their states during two consecutive timestamps. Therefore, we propose several incremental angle-based outlier detection approaches over data streams based on ABOD and its variants that provide visible speed-up without loss of accuracy. Firstly, the basic ideas of these incremental algorithms are introduced. Then, we explain the time complexity of them. Finally, we use synthetic data streams to prove their efficiency.

Continuous Angle-based Outlier Detection on High-dimensional Data Streams

Uncertain data are common due to the increasing usage of sensors, radio frequency identification (RFID), GPS and similar devices for data collection. The causes of uncertainty include limitations of measurements, inclusion of noise, inconsistent supply voltage and delay or loss of data in transfer. In order to manage, query or mine such data, data uncertainty needs to be considered. Hence, this paper studies the problem of top-k distance-based outlier detection from uncertain data objects. In this work, an uncertain object is modelled by a probability density function of a Gaussian distribution. The naive approach of distance-based outlier detection makes use of nested loop. This approach is very costly due to the expensive distance function between two uncertain objects. Therefore, a populated-cells list (PC-list) approach of outlier detection is proposed. Using the PC-list, the proposed top-k outlier detection algorithm needs to consider only a fraction of dataset objects and hence quickly identifies candidate objects for top-k outliers. Two approximate top-k outlier detection algorithms are presented to further increase the efficiency of the top-k outlier detection algorithm. An extensive empirical study on synthetic and real datasets is also presented to prove the accuracy, efficiency and scalability of the proposed algorithms.

/pdf/top-k-outlier-detection-from-uncertain-data-4bv8462r3s.pdf

Top-k Outlier Detection from Uncertain Data

This paper proposes a novel approach for querying large-scale XML data using PC cluster system. With the recent spread of the XML format, large-scale data coded in XML ranging from several hundreds of megabytes to several gigabytes has become common. However, XML databases are often innefficient in dealing with huge XML data. The problem is the complexity of the XML data model and query processing. To cope with this problem, we attempt to construct a parallel XML database on top of a PC cluster system. To this end, we discuss XML data partitioning to enable parallel processing of XML queries. We introduce a path-based partitioning for XML data. The obtained XML fragments are then allocated to cluster nodes. To obtain cost-efficient allocation of the fragments, we discuss cost functions for parallel XPath processing and an algorithm to compute pseudo-optimal allocation, which is based on the well-known genetic algorithm. Finally, we demonstrate effectiveness of the proposed scheme by a series of experiments.

Querying XML Data using PC Cluster System

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Hiroyuki Kitagawa

Author Tools

Chat about Author