Proceedings Article10.1109/SRDS51746.2020.00027
Anomaly Detection via Mining Numerical Workflow Relations from Logs
Bo Zhang,Hongyu Zhang,Pablo Moscato,Aozhong Zhang +3 more
- 26 Jun 2020
- pp 195-204
29
TL;DR: A novel approach named ADR (stands for Anomaly Detection by workflow Relations), which employs matrix nullspace to mine numerical relations from log data, which is effective for both offline and online anomaly detection.
read more
Abstract: Complex software-intensive systems, especially distributed systems, generate logs for troubleshooting The logs are text messages recording system events, which can help engineers determine the system’s runtime status This paper proposes a novel approach named ADR (stands for Anomaly Detection by workflow Relations), which employs matrix nullspace to mine numerical relations from log data The mined relations can be used for both offline and online anomaly detection and facilitate fault diagnosis We have evaluated ADR on log data collected from two distributed systems ADR successfully mined 87 and 669 numerical relations from the logs and used them to detect anomalies with high precision and recall For online anomaly detection, ADR employs PSO (Particle Swarm Optimization) to find the optimal sliding windows’ size and achieves fast anomaly detection The experimental results confirm that ADR is effective for both offline and online anomaly detection
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Log-based Anomaly Detection with Deep Learning: How Far Are We?
Van-Hoang Le,Hongyu Zhang +1 more
- 09 Feb 2022
TL;DR: An in-depth analysis of five state-of-the-art deep learning-based models for detecting system anomalies on four public log datasets, focusing on several aspects of model evaluation, including training data selection, data grouping, class distribution, data noise, and early detection ability.
Log Parsing with Prompt-based Few-shot Learning
Van-Hoang Le,Hongyu Zhang +1 more
- 15 Feb 2023
TL;DR: In this article , a prompt-based few-shot learning method is proposed to capture the patterns of log templates using a few labeled log data, and an adaptive random sampling algorithm is designed to select a small yet diverse training set.
34
LLMParser: An Exploratory Study on Using Large Language Models for Log Parsing
Zeyang Ma,An Ran Chen,Dong Jae Kim,Tse-Husn Chen,Shaowei Wang +4 more
- 12 Apr 2024
TL;DR: This study provides empirical evidence for using LLMs for log parsing and highlights the limitations and future research direction of LLM-based log parsers.
24
LogGD: Detecting Anomalies from System Logs with Graph Neural Networks
01 Dec 2022
TL;DR: Wang et al. as discussed by the authors proposed a graph-based log anomaly detection method, LogGD, to effectively address the issue by transforming log sequences into graphs, which combines graph structure and node semantics for log-based anomaly detection.
18
Semi-supervised and unsupervised anomaly detection by mining numerical workflow relations from system logs
Hongyu Zhang,Van-Hoang Le,Pablo Moscato,Aozhong Zhang +3 more
- 03 Dec 2022
TL;DR: The experimental results show that ADR can extract the workflow relations from log data, and is effective for log-based anomaly detection in both semi-supervised and unsupervised manners.
15
References
Particle swarm optimization
James Kennedy,Russell C. Eberhart +1 more
- 06 Aug 2002
TL;DR: A concept for the optimization of nonlinear functions using particle swarm methodology is introduced, and the evolution of several paradigms is outlined, and an implementation of one of the paradigm is discussed.
44.1K
Particle Swarm Optimization.
James Kennedy
- 01 Jan 2017
TL;DR: A concept for the optimization of nonlinear functions using particle swarm methodology is introduced, and the evolution of several paradigms is outlined, and an implementation of one of the paradigm is discussed.
35K
Particle swarm optimization
TL;DR: A snapshot of particle swarming from the authors’ perspective, including variations in the algorithm, current and ongoing research, applications and open problems, is included.
Comparison of the predicted and observed secondary structure of T4 phage lysozyme.
TL;DR: Although empirical predictions based on larger numbers of known protein structure tend to be more accurate than those based on a limited sample, the improvement in accuracy is not dramatic, suggesting that the accuracy of current empirical predictive methods will not be substantially increased simply by the inclusion of more data from additional protein structure determinations.
5.4K
•Journal Article
On linear algebra and its applications
V. Yegnanarayanan,S. Sreekumar +1 more
TL;DR: In this paper, the modified Jordan Exchange method is introduced and a new twist to the Jordan exchange method is given, and the resulting technique modified JE method is used for proving different pertinent results concerning simplex method of linear programming.
1.6K