A PCA-Based Change Detection Framework for Multidimensional Data Streams: Change Detection in Multidimensional Data Streams
Abdulhakim Qahtan,Basma Alharbi,Suojin Wang,Xiangliang Zhang +3 more
- 10 Aug 2015
- pp 935-944
TL;DR: This paper proposes a framework for detecting changes in multidimensional data streams based on principal component analysis, which is used for projecting data into a lower dimensional space, thus facilitating density estimation and change-score calculations and has advantages over existing approaches.
read more
Abstract: Detecting changes in multidimensional data streams is an important and challenging task. In unsupervised change detection, changes are usually detected by comparing the distribution in a current (test) window with a reference window. It is thus essential to design divergence metrics and density estimators for comparing the data distributions, which are mostly done for univariate data. Detecting changes in multidimensional data streams brings difficulties to the density estimation and comparisons. In this paper, we propose a framework for detecting changes in multidimensional data streams based on principal component analysis, which is used for projecting data into a lower dimensional space, thus facilitating density estimation and change-score calculations.The proposed framework also has advantages over existing approaches by reducing computational costs with an efficient density estimator, promoting the change-score calculation by introducing effective divergence metrics, and by minimizing the efforts required from users on the threshold parameter setting by using the Page-Hinkley test. The evaluation results on synthetic and real data show that our framework outperforms two baseline methods in terms of both detection accuracy and computational costs.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Learning under Concept Drift: A Review
TL;DR: A high quality, instructive review of current research developments and trends in the concept drift field is conducted, and a framework of learning under concept drift is established including three main components: concept drift detection, concept drift understanding, and concept drift adaptation.
995
Learning under Concept Drift: A Review
TL;DR: In this paper, the authors present a review of the recent research in the field of concept drift and propose a framework of learning under concept drift. But, the focus of this survey is on the detection, understanding and adaptation of the concept drift in streaming data.
752
A survey on data preprocessing for data stream mining
TL;DR: This survey summarizes, categorize and analyze those contributions on data preprocessing that cope with streaming data, and takes into account the existing relationships between the different families of methods (feature and instance selection, and discretization).
483
Clinical artificial intelligence quality improvement: towards continual monitoring and updating of AI algorithms in healthcare
Jean Feng,Rachael V. Phillips,Ivana Malenica,Andrew M. Bishara,Alan E. Hubbard,Leo Anthony Celi,R. Pirracchio +6 more
TL;DR: In this article , the authors advocate for the creation of hospital units responsible for quality assurance and improvement of these algorithms, which they refer to as "AI-QI" units, and discuss how tools that have long been used in hospital quality assurance, quality improvement can be adapted to monitor static ML algorithms.
On the reliable detection of concept drift from streaming unlabeled data
TL;DR: The Margin Density Drift Detection (MD3) algorithm, which tracks the number of samples in the uncertainty region of a classifier, as a metric to detect drift, is proposed, which leads to a detection scheme which is credible, label efficient and general in its applicability.
179
References
Activity recognition using cell phone accelerometers
TL;DR: This work describes and evaluates a system that uses phone-based accelerometers to perform activity recognition, a task which involves identifying the physical activity a user is performing, and has a wide range of applications, including automatic customization of the mobile device's behavior based upon a user's activity.
Comprehensive Survey on Distance/Similarity Measures between Probability Density Functions
Sung-Hyuk Cha
- 01 Jan 2007
TL;DR: Various distance/similarity measures that are applicable to compare two probability density functions, pdf in short, are reviewed and categorized in both syntactic and semantic relationships to reveal similarities among numerous distance/Similarity measures.
1.9K
•Proceedings Article
Learning from Time-Changing Data with Adaptive Windowing
Albert Bifet,Ricard Gavaldà +1 more
- 01 Jan 2007
TL;DR: A new approach for dealing with distribution change and concept drift when learning from data sequences that may vary with time is presented, using sliding windows whose size is recomputed online according to the rate of change observed from the data in the window itself.
Detecting change in data streams
Daniel Kifer,Shai Ben-David,Johannes Gehrke +2 more
- 31 Aug 2004
TL;DR: A novel method for the detection and estimation of change that assumes that the points in the stream are independently generated, but otherwise makes no assumptions on the nature of the generating distribution.
On-Line Unsupervised Outlier Detection Using Finite Mixtures with Discounting Learning Algorithms
TL;DR: An experimental application to network intrusion detection shows that SmartSifter was able to identify data with high scores that corresponded to attacks, with low computational costs.
672