Bridging the divide between software developers and operators using logs
Weiyi Shang
- 02 Jun 2012
- pp 1583-1586
TL;DR: The use of logs are proposed as mechanism to bridge the gap between the software development and operation worlds and the value of logs is demonstrated as a tool to support developers and operators.
read more
Abstract: There is a growing gap between the software development and operation worlds. Software developers rarely divulge development knowledge about the software to operators, while operators rarely communicate field knowledge to developers. To improve the quality and reduce the operational cost of large-scale software systems, bridging the gap between these two worlds is essential. This thesis proposes the use of logs as mechanism to bridge the gap between these two worlds. Logs are messages generated from statements inserted by developers in the source code and are often used by operators for monitoring the field operation of a system. However, the rich knowledge in logs has not yet been fully used because of their non-structured nature, their large scale, and the use of the ad hoc log analysis techniques. Through case studies on large commercial and open source systems, we plan to demonstrate the value of logs as a tool to support developers and operators.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
A qualitative study of DevOps usage in practice
TL;DR: The ways in which organizations implement DevOps and the outcomes they experience are described and it is observed that all organizations were positive about their experiences and only minor problems were encountered while adopting DevOps.
221
Dimensions of DevOps
Lucy Ellen Lwakatare,Pasi Kuvaja,Markku Oivo +2 more
- 25 May 2015
TL;DR: This study investigates the elements that characterize the Dev Ops phenomenon using a literature survey and interviews with practitioners actively involved in the DevOps movement to develop an initial conceptual framework.
176
•Posted Content
A Survey on Automated Log Analysis for Reliability Engineering.
TL;DR: This survey presents a detailed overview of automated log analysis research, including how to automate and assist the writing of logging statements, how to compress logs,How to parse logs into structured event templates, and how to employ logs to detect anomalies, predict failures, and facilitate diagnosis.
Examining the stability of logging statements
TL;DR: This work uses a random forest classifier to determine whether a just-introduced logging statement will change in the future, based solely on metrics that are calculated when it is introduced, and examines whether a long-lived logging statement is likely to change based on its change history.
122
LogClass: Anomalous Log Identification and Classification With Partial Labels
Weibin Meng,Ying Liu,Shenglin Zhang,Federico Zaiter,Yuzhe Zhang,Yuheng Huang,Zhaoyang Yu,Yuzhi Zhang,Lei Song,Ming Zhang,Dan Pei +10 more
TL;DR: This work proposes LogClass, a framework to automatically and robustly identify and classify anomalous logs for network and service based on partial labels, which combines a word representation method, a positive and unlabeled learning (PU learning) model, and a machine learning classifier.
52
References
Detecting large-scale system problems by mining console logs
Wei Xu,Ling Huang,Armando Fox,David A. Patterson,Michael I. Jordan +4 more
- 11 Oct 2009
TL;DR: In this article, a general methodology to mine this rich source of information to automatically detect system runtime problems was proposed, combining source code analysis with information retrieval to create composite features and then analyze these features using machine learning to detect operational problems.
1K
•Proceedings Article
Detecting Large-Scale System Problems by Mining Console Logs
Wei Xu,Ling Huang,Armando Fox,David A. Patterson,Michael I. Jordan +4 more
- 21 Jun 2010
TL;DR: This work first parse console logs by combining source code analysis with information retrieval to create composite features, and then analyzes these features using machine learning to detect operational problems to automatically detect system runtime problems.
On the Calculus of Relations
TL;DR: The logical theory which is called the calculus of (binary) relations, and which will constitute the subject of this paper, has had a strange and rather capricious line of historical development.
Software Reflexion Models: Bridging the Gap Between Source and High-Level Models
Gail C. Murphy,David Notkin,Kevin Sullivan +2 more
- 01 Jan 1995
TL;DR: In this article, an approach that helps an engineer use a high-level model of the structure of an existing software system as a lens through which to see a model of that system's source code is presented.
484
Leveraging existing instrumentation to automatically infer invariant-constrained models
Ivan Beschastnikh,Yuriy Brun,Sigurd Schneider,Michael Sloan,Michael D. Ernst +4 more
- 09 Sep 2011
TL;DR: It is formally prove that Synoptic always produces a model that satisfies exactly the temporal invariants mined from the log, and it is argued that it does so efficiently.