Journal Article10.1109/TDSC.2009.4
A Large-Scale Study of Failures in High-Performance Computing Systems
Bianca Schroeder,Garth A. Gibson +1 more
906
TL;DR: Analysis of failure data collected at two large high-performance computing sites finds that average failure rates differ wildly across systems, ranging from 20-1000 failures per year, and that time between failures is modeled well by a Weibull distribution with decreasing hazard rate.
read more
Abstract: Designing highly dependable systems requires a good understanding of failure characteristics. Unfortunately, little raw data on failures in large IT installations are publicly available. This paper analyzes failure data collected at two large high-performance computing sites. The first data set has been collected over the past nine years at Los Alamos National Laboratory (LANL) and has recently been made publicly available. It covers 23,000 failures recorded on more than 20 different systems at LANL, mostly large clusters of SMP and NUMA nodes. The second data set has been collected over the period of one year on one large supercomputing system comprising 20 nodes and more than 10,000 processors. We study the statistics of the data, including the root cause of failures, the mean time between failures, and the mean time to repair. We find, for example, that average failure rates differ wildly across systems, ranging from 20-1000 failures per year, and that time between failures is modeled well by a Weibull distribution with decreasing hazard rate. From one system to another, mean repair time varies from less than an hour to more than a day, and repair times are well modeled by a lognormal distribution.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Stochastic programming for selective maintenance optimization with uncertainty in the next mission conditions
Milad Ghorbani,Mustapha Nourelfath,Michel Gendreau +2 more
- 01 Sep 2023
TL;DR: This study applies two-stage stochastic programming to optimize selective maintenance for multi-component systems with uncertain mission conditions, incorporating uncertainties in operating conditions and mission time to ensure mission success and minimize system failure.
6
Portable Topology-Aware MPI-I/O
Robert Latham,Leonardo Bautista-Gomez,Pavan Balaji +2 more
- 01 Dec 2017
TL;DR: This paper proposes extensions to MPI enabling codes to determine which nodes of a system share common features, which provides a portable mechanism for resource discovery and lays the foundation for additional optimizations in checkpointing and in ROMIO.
6
Design and Implementation of High-availability PaaS Platform Based on Virtualization Platform
Zepeng Wen,Yan Liang,Gongliang Li +2 more
- 12 Jun 2020
TL;DR: This paper proposes high-availability design of operating components, resource pools and middleware of the PaaS platform, and at the same time maps the design to the physical deployment mode, the experimental results show that the design is feasible and effective.
6
Node anomaly detection for homogeneous distributed environments
TL;DR: This paper introduces a new type of anomalies called contextual collective anomaly, and proposes a framework to discover this type of anomaly over a collection of data streams in real time that can accurately identify the anomalies by taking both the contextual information and the historical information of a data stream into consideration.
5
Modeling and Analyzing the Dependability of Short Range Wireless Technologies via Field Failure Data Analysis
TL;DR: Through the experience gained from extensive failure analysis of Bluetooth networks, the paper shows how field failure data can play a key role to fill the gap on understanding the dependability behavior of wireless networks.
5
References
Introduction to Probability Models.
A. Csenki,Sheldon M. Ross +1 more
TL;DR: Download Introduction to Probability Models Sheldon M Download Pdf octave levenspiel solution manual pdf stochastic processes sheldon m ross pdf.
3.9K
Self-similarity through high-variability: statistical analysis of Ethernet LAN traffic at the source level
TL;DR: In this article, the authors provide a plausible physical explanation for the occurrence of self-similarity in local-area network (LAN) traffic, based on convergence results for processes that exhibit high variability and is supported by detailed statistical analyzes of real-time traffic measurements from Ethernet LANs at the level of individual sources.
Self-similarity through high-variability: statistical analysis of ethernet LAN traffic at the source level
Walter Willinger,Murad S. Taqqu,Robert Sherman,Daniel V. Wilson +3 more
- 01 Oct 1995
TL;DR: This paper provides a plausible physical explanation for the occurrence of self-similarity in high-speed network traffic based on convergence results for processes that exhibit high variability and is supported by detailed statistical analyses of real-time traffic measurements from Ethernet LAN's at the level of individual sources.
1.1K
•Proceedings Article
Disk failures in the real world: what does an MTTF of 1,000,000 hours mean to you?
Bianca Schroeder,Garth A. Gibson +1 more
- 13 Feb 2007
TL;DR: In this article, the authors present and analyze field-gathered disk replacement data from a number of large production systems, including high-performance computing sites and internet services sites, and find that in the field, annual disk replacement rates typically exceed 1%, with 2-4% common and up to 13% observed on some systems.
Why do computers stop and what can be done about it
Jim Gray
- 01 Jan 1985
TL;DR: It is pointed out that faults in production software are often soft (transient) and that a ransaction mechanism combined with persistent processpairs provides fault-tolerant execution -- the key to software fault -tolerance.