Abstract: Automatic program repair has been a longstanding goal in software engineering, yet debugging remains a largely manual process. We introduce a fully automated method for locating and repairing bugs in software. The approach works on off-the-shelf legacy applications and does not require formal specifications, program annotations or special coding practices. Once a program fault is discovered, an extended form of genetic programming is used to evolve program variants until one is found that both retains required functionality and also avoids the defect in question. Standard test cases are used to exercise the fault and to encode program requirements. After a successful repair has been discovered, it is minimized using structural differencing algorithms and delta debugging. We describe the proposed method and report experimental results demonstrating that it can successfully repair ten different C programs totaling 63,000 lines in under 200 seconds, on average.

Abstract: Reproducing bugs is hard. Deterministic replay systems address this problem by providing a high-fidelity replica of an original program run that can be repeatedly executed to zero-in on bugs. Unfortunately, existing replay systems for multiprocessor programs fall short. These systems either incur high overheads, rely on non-standard multiprocessor hardware, or fail to reliably reproduce executions. Their primary stumbling block is data races -- a source of nondeterminism that must be captured if executions are to be faithfully reproduced.In this paper, we present ODR--a software-only replay system that reproduces bugs and provides low-overhead multiprocessor recording. The key observation behind ODR is that, for debugging purposes, a replay system does not need to generate a high-fidelity replica of the original execution. Instead, it suffices to produce any execution that exhibits the same outputs as the original. Guided by this observation, ODR relaxes its fidelity guarantees to avoid the problem of reproducing data-races altogether. The result is a system that replays real multiprocessor applications, such as Apache, MySQL, and the Java Virtual Machine, and provides low record-mode overhead.

Abstract: Current shared memory multicore and multiprocessor systems are nondeterministic. Each time these systems execute a multithreaded application, even if supplied with the same input, they can produce a different output. This frustrates debugging and limits the ability to properly test multithreaded code, becoming a major stumbling block to the much-needed widespread adoption of parallel programming.In this paper we make the case for fully deterministic shared memory multiprocessing (DMP). The behavior of an arbitrary multithreaded program on a DMP system is only a function of its inputs. The core idea is to make inter-thread communication fully deterministic. Previous approaches to coping with nondeterminism in multithreaded programs have focused on replay, a technique useful only for debugging. In contrast, while DMP systems are directly useful for debugging by offering repeatability by default, we argue that parallel programs should execute deterministically in the field as well. This has the potential to make testing more assuring and increase the reliability of deployed multithreaded software. We propose a range of approaches to enforcing determinism and discuss their implementation trade-offs. We show that determinism can be provided with little performance cost using our architecture proposals on future hardware, and that software-only approaches can be utilized on existing systems.

Abstract: Windows Error Reporting (WER) is a distributed system that automates the processing of error reports coming from an installed base of a billion machines. WER has collected billions of error reports in ten years of operation. It collects error data automatically and classifies errors into buckets, which are used to prioritize developer effort and report fixes to users. WER uses a progressive approach to data collection, which minimizes overhead for most reports yet allows developers to collect detailed information when needed. WER takes advantage of its scale to use error statistics as a tool in debugging; this allows developers to isolate bugs that could not be found at smaller scale. WER has been designed for large scale: one pair of database servers can record all the errors that occur on all Windows computers worldwide.

Abstract: Turing Lecture from the winners of the 2007 ACM A.M. Turing Award.In 1981, Edmund M. Clarke and E. Allen Emerson, working in the USA, and Joseph Sifakis working independently in France, authored seminal papers that founded what has become the highly successful field of model checking. This verification technology provides an algorithmic means of determining whether an abstract model---representing, for example, a hardware or software design---satisfies a formal specification expressed as a temporal logic (TL) formula. Moreover, if the property does not hold, the method identifies a counterexample execution that shows the source of the problem.The progression of model checking to the point where it can be successfully used for complex systems has required the development of sophisticated means of coping with what is known as the state explosion problem. Great strides have been made on this problem over the past 28 years by what is now a very large international research community. As a result many major hardware and software companies are beginning to use model checking in practice. Examples of its use include the verification of VLSI circuits, communication protocols, software device drivers, real-time embedded systems, and security algorithms.The work of Clarke, Emerson, and Sifakis continues to be central to the success of this research area. Their work over the years has led to the creation of new logics for specification, new verification algorithms, and surprising theoretical results. Model checking tools, created by both academic and industrial teams, have resulted in an entirely novel approach to verification and test case generation. This approach, for example, often enables engineers in the electronics industry to design complex systems with considerable assurance regarding the correctness of their initial designs. Model checking promises to have an even greater impact on the hardware and software industries in the future. ---Moshe Y. Vardi, Editor-in-Chief

Abstract: A CPU emulator is a software that simulates a hardware CPU. Emulators are widely used by computer scientists for various kind of activities (e.g., debugging, profiling, and malware analysis). Although no theoretical limitation prevents to develop an emulator that faithfully emulates a physical CPU, writing a fully featured emulator is a very challenging and error-prone task. Modern CISC architectures have a very rich instruction set, some instructions lack proper specifications, and others may have undefined effects in corner-cases. This paper presents a testing methodology specific for CPU emulators, based on fuzzing. The emulator is "stressed" with specially crafted test-cases, to verify whether the CPU is properly emulated or not. Improper behaviours of the emulator are detected by running the same test-case concurrently on the emulated and on the physical CPUs and by comparing the state of the two after the execution. Differences in the final state testify defects in the code of the emulator. We implemented this methodology in a prototype (codenamed EmuFuzzer), analysed four state-of-the-art IA-32 emulators (QEMU, Valgrind, Pin and BOCHS), and found several defects in each of them, some of which can prevent the proper execution of programs.

Abstract: Advances in recent years have made it possible in some cases to locate a bug (the source of a failure) automatically. But debugging is also about correcting bugs. Can tools do this automatically? The results reported in this paper, from the new PACHIKA tool, suggest that such a goal may be reachable. PACHIKA leverages differences in program behavior to generate program fixes directly. It automatically summarizes executions to object behavior models, determines differences between passing and failing runs, generates possible fixes, and assesses them via the regression test suite. Evaluated on the ASPECTJ bug history, PACHIKA generates a valid fix for 3 out of 18 crashing bugs; each fix pinpoints the bug location and passes the ASPECTJ test suite.

Abstract: Debugging and diagnostic tools are some of the most important software development tools, but most expect developers choose the right code to inspect. Unfortunately, this rarely occurs. A new tool called the Whyline is described which avoids such speculation by allowing developers to select questions about a program's output. The tool then helps developers work backwards from output to its causes. The prototype, which supports Java programs, was evaluated in an experiment in which participants investigated two real bug reports from an open source project using either the Whyline or a breakpoint debugger. Whyline users were successful about three times as often and about twice as fast compared to the control group, and were extremely positive about the tool's ability to simplify diagnostic tasks in software development work.

Abstract: In large software development projects, when a programmer is assigned a bug to fix, she typically spends a lot of time searching (in an ad-hoc manner) for instances from the past where similar bugs have been debugged, analyzed and resolved. Systematic search tools that allow the programmer to express the context of the current bug, and search through diverse data repositories associated with large projects can greatly improve the productivity of debugging This paper presents the design, implementation and experience from such a search tool called DebugAdvisor. The context of a bug includes all the information a programmer has about the bug, including natural language text, textual rendering of core dumps, debugger output etc. Our key insight is to allow the programmer to collate this entire context as a query to search for related information. Thus, DebugAdvisor allows the programmer to search using a fat query, which could be kilobytes of structured and unstructured data describing the contextual information for the current bug. Information retrieval in the presence of fat queries and variegated data repositories, all of which contain a mix of structured and unstructured data is a challenging problem. We present novel ideas to solve this problem. We have deployed DebugAdvisor to over 100 users inside Microsoft. In addition to standard metrics such as precision and recall, we present extensive qualitative and quantitative feedback from our users.

Abstract: This is the first book on parallel MATLAB and the first parallel computing book focused on the design, code, debug, and test techniques required to quickly produce well-performing parallel programs. MATLAB is currently the dominant language of technical computing with one million users worldwide, many of whom can benefit from the increased power offered by inexpensive multicore and multinode parallel computers. MATLAB is an ideal environment for learning about parallel computing, allowing the user to focus on parallel algorithms instead of the details of implementation. Parallel MATLAB for Multicore and Multinode Computers covers more parallel algorithms and parallel programming models than any other parallel programming book due to the succinctness of MATLAB. It presents a hands-on approach with numerous example programs; wherever possible, the examples are drawn from widely known and well-documented parallel benchmark codes that are representative of many real applications across the field of technical computing. Audience: Intended for professional scientists and engineers, as well as undergraduate or graduate students, who use MATLAB. It is suitable as either the primary book in a parallel computing class or as a supplementary text in a numerical computing class or a computer science algorithms class. Contents: List of Figures; List of Tables; List of Algorithms; Preface; Acknowledgments; Part I: Fundamentals: Chapter 1: Primer: Notation and Interfaces; Chapter 2: Introduction to pMatlab; Chapter 3: Interacting with Distributed Arrays; Part II: Advanced Techniques: Chapter 4: Parallel Programming Models; Chapter 5: Advanced Distributed Array Programming; Chapter 6: Performance Metrics and Software Architecture; Part III: Case Studies: Chapter 7: Parallel Application Analysis; Chapter 8: Stream; Chapter 9: RandomAccess; Chapter 10: Fast Fourier Transform; Chapter 11: High Performance Linpack; Appendix: Notation for Hierarchical Parallel Multicore Algorithms; Index

Abstract: This book is an in-depth introduction to Erlang, a programming language ideal for any situation where concurrency, fault tolerance, and fast response is essential. Erlang is gaining widespread adoption with the advent of multi-core processors and their new scalable approach to concurrency. With this guide you'll learn how to write complex concurrent programs in Erlang, regardless of your programming background or experience. Written by leaders of the international Erlang community -- and based on their training material -- Erlang Programming focuses on the language's syntax and semantics, and explains pattern matching, proper lists, recursion, debugging, networking, and concurrency. This book helps you: Understand the strengths of Erlang and why its designers included specific features Learn the concepts behind concurrency and Erlang's way of handling it Write efficient Erlang programs while keeping code neat and readable Discover how Erlang fills the requirements for distributed systems Add simple graphical user interfaces with little effort Learn Erlang's tracing mechanisms for debugging concurrent and distributed systems Use the built-in Mnesia database and other table storage features Erlang Programming provides exercises at the end of each chapter and simple examples throughout the book.

Abstract: Most system level software is written in C and executed concurrently. Because such software is often critical for system reliability, it is an ideal target for formal verification. Annotated C and the Verified C Compiler (VCC) form the first modular sound verification methodology for concurrent C that scales to real-world production code. VCC is integrated in Microsoft Visual Studio and it comes with support for verification debugging: an explorer for counter-examples of failed proofs helps to find errors in code or specifications, and a prover log analyzer helps debugging proof attempts that exhaust available resources (memory, time). VCC is currently used to verify the core of Microsoft Hyper-V, consisting of 50,000 lines of system-level C code.

Abstract: Locating software components which are responsible for observed failures is the most expensive, error-prone phase in the software development life cycle. Automated diagnosis of software faults can improve the efficiency of the debugging process, and is therefore an important process for the development of dependable software. In this paper we present a toolset for automatic fault localization, dubbed Zoltar, which hosts a range of spectrum-based fault localization techniques featuring BARINEL, our latest algorithm. The toolset provides the infrastructure to automatically instrument the source code of software programs to produce runtime data, which is subsequently analyzed to return a ranked list of diagnosis candidates. Aimed at total automation (e.g., for runtime fault diagnosis), Zoltar has the capability of instrumenting the program under analysis with fault screeners as a run-time replacement for design-time test oracles.

Abstract: One of the major challenges in managing networked and dynamic ontologies is to handle inconsistencies in single ontologies, and inconsistencies introduced by integrating multiple distributed ontologies. Our RaDON system provides functionalities to repair and diagnose ontology networks by extending the capabilities of existing reasoners. The system integrates several new debugging and repairing algorithms, such as a relevance-directed algorithm to meet the various needs of the users.

Abstract: Discovering end-to-end request-processing paths is crucial in many modern IT environments for reasons varying from debugging and bottleneck analysis to billing and auditing. Existing solutions for this problem fall into two broad categories: statistical inference and intrusive instrumentation. The statistical approaches infer request-processing paths in a "most likely" way and their accuracy degrades as the workload increases. The instrumentation approaches can be accurate, but they are system dependent as they require knowledge (and often source code) of the application as well as time and effort from skilled programmers. We have developed a discovery technique called vPath that overcomes these shortcomings. Unlike techniques using statistical inference, vPath provides precise path discovery, by monitoring thread and network activities and reasoning about their causality. Unlike techniques using intrusive instrumentation, vPath is implemented in a virtual machine monitor, making it agnostic of the overlying middleware or application. Our evaluation using a diverse set of applications (TPC-W, RUBiS, MediaWiki, and the home-grown vApp) written in different programming languages (C, Java, and PHP) demonstrates the generality and accuracy of vPath as well as its low overhead. For example, turning on vPath affects the throughput and response time of TPC-W by only 6%.

Abstract: This article presents TOD (trace oriented debugger), a prototype scalable omniscient debugger for Java, which aims at making omniscient debugging practical, at last. Omniscient debuggers, also known as back-in-time or reversible debuggers, record the whole history, or execution trace, of a debugged program and let the user freely explore it. This approach combines the advantages of both log-based (past activity is never lost) and breakpoint based debugging (interactive navigation, step-by-step execution, and complete stack inspection). Omniscient debuggers simulate step-by-step execution both forward and backward, avoiding having to rerun the whole program many times to pinpoint the bug's root cause. More importantly, they make it possible to navigate through the history of a program by following causal links, so questions that would otherwise require a significant effort can be answered instantly for instance, "When was variable x assigned a null value?" or "What was the state of object o when it was passed as an argument to method foo?".

Abstract: Deriving a formal specification from an informal design intent is an error-prone process. The resulting specification may be incomplete, unrealizable, or in conflict with the design intent. We propose a debugging method for incorrect specifications that does not need an implementation. We show that we can explain conflicts with the design intent by explaining unrealizability. Our approach for explaining unrealizability is based on counterstrategies. Since counterstrategies may be large, we propose several ways to simplify them. First, we simplify the specification itself by removing both requirements and variables that do not contribute to the problem. Second, we heuristically search for a countertrace, i.e., a single input trace that suffices to demonstrate unrealizability. Finally, we present the countertrace or the counterstrategy to the user in extensive form as a graph and implicitly as an interactive game. We present experimental results for specifications given as GR(1) formulas.

Abstract: This paper proposes a new parallel execution model where programmers augment a sequential program with pieces of code called serializers that dynamically map computational operations into serialization sets of dependent operations. A runtime system executes operations in the same serialization set in program order, and may concurrently execute operations in different sets. Because serialization sets establish a logical ordering on all operations, the resulting parallel execution is predictable and deterministic.We describe the API and design of Prometheus, a C++ library that implements the serialization set abstraction through compile-time template instantiation and a runtime support library. We evaluate a set of parallel programs running on the x86_64 and SPARC-V9 instruction sets and study their performance on multicore, symmetric multiprocessor, and ccNUMA parallel machines. By contrast with conventional parallel execution models, we find that Prometheus programs are significantly easier to write, test, and debug, and their parallel execution achieves comparable performance.

Abstract: Message passing is a very popular style of parallel programming, used in a wide variety of applications and supported by many APIs, such as BSD sockets, MPI and PVM. Its importance has motivated significant amounts of research on optimization and debugging techniques for such applications. Although this work has produced impressive results, it has also failed to fulfill its full potential. The reason is that while prior work has focused on runtime techniques, there has been very little work on compiler analyses that understand the properties of parallel message passing applications and use this information to improve application performance and quality of debuggers.This paper presents a novel compiler analysis framework that extends dataflow to parallel message passing applications on arbitrary numbers of processes. It works on an extended control-flow graph that includes all possible inter-process interactions of any numbers of processes. This enables dataflow analyses built on top of this framework to incorporate information about the application's parallel behavior and communication topology. The parallel dataflow framework can be instantiated with a variety of specific dataflow analyses as well as abstractions that can tune the accuracy of communication topology detection against its cost.The proposed framework bridges the gap between prior work on parallel runtime systems and sequential dataflow analyses, enabling new transformations, runtime optimizations and bug detection tools that require knowledge of the application's communication topology. We instantiate this framework with two different symbolic analyses and show how these analyses can detect different types of communication patterns, which enables the use of dataflow analyses on a wide variety of real applications.

Abstract: This article describes the Open Source SCA Implementation::Embedded project, an open source software development kit designed for rapid prototyping of software defined radios consistent with the software communications architecture. The SCA is a product of the American military's SDR acquisition program and has played a large role in SDR development in the military and in the wireless industry. OSSIE was designed for use in wireless communications curricula and research efforts, so it is easy to learn and illustrative of software engineering, programming, and communications engineering concepts important in industrial practice today. OSSIE includes a core framework (i.e., common services enumerated in the SCA). It also includes graphical user interface-oriented tools that are easily learned and free to download and use. The tools auto-generate SCA-specific component source code and supporting files, leaving the developer to provide signal processing functionality. In addition, visualization tools for debugging and a growing library of SDR software components are available. Discussed herein are examples of SDRs designed using OSSIE, including embedded applications. OSSIE enables easy transition from concepts to practice in SDR design for communications engineers who may not have a strong software background.

Abstract: Post-silicon validation is an essential step in the design flow, which is needed to demonstrate that the implemented circuit meets its intended behavior. Due to lack of in-system controllability and observability, design-for-debug hardware is employed to aid post-silicon validation. A number of solutions have been proposed to implement the design-for-debug hardware, as well as to analyze the debug data that is acquired. Although the design entry is done at the register-transfer level, the existing approaches to aid post-silicon validation rely primarily on the information extracted from the gate level circuit descriptions. We anticipate that, as the design complexity continues to grow, extracting and processing circuit information at this level will become increasingly difficult. In this paper, we briefly summarize the known art and discuss some possible directions of investigation that can utilize high-level circuit descriptions to augment the existing solutions.

Abstract: Software reliability growth models based on a nonhomogeneous Poisson process (NHPP) have been considered as one of the most effective among various models since they integrate the information regarding testing and debugging activities observed in the testing phase into the software reliability model. Although most of the existing NHPP models have progressed successfully in their estimation/prediction accuracies by modifying the assumptions with regard to the testing process, these models were developed based on the instantaneous fault-removal assumption. In this paper, we develop a generalized NHPP software reliability model considering quasi-renewal time-delay fault removal. The quasi-renewal process is employed to estimate the time delay due to identifying and prioritizing the detected faults before actual code change in the software reliability assessment. Model formulation based on the quasi-renewal time-delay assumption is provided, and the generalized mean value function (MVF) for the proposed model is derived by using the method of steps. The general solution of the MVFs for the proposed model is also obtained for some specific existing models. The numerical examples, based on a software failure data set, show that the consideration of quasi-renewal time-delay fault-removal assumption improves the descriptive properties of the model, which means that the length of time delay is getting decreased since testers and programmers adapt themselves to the working environment as testing and debugging activities are in progress.

Abstract: Part I. Essential Steps: 1. Problem formulation 2. Software implementation: what to solve 3. Software implementation: how to solve (preprocessing) 4. Software implementation: visualizing and manipulating solution (postprocessing) 5. Validation, sensitivity analysis, optimization and debugging Part II. Case Studies: 6. Case studies Part III. Background Material: 7. Governing equations and boundary conditions 8. Source terms 9. Material properties and other input parameters 10. Solving the equations: numerical methods.

Abstract: We present a scalable temporal order analysis technique that supports debugging of large scale applications by classifying MPI tasks based on their logical program execution order. Our approach combines static analysis techniques with dynamic analysis to determine this temporal order scalably. It uses scalable stack trace analysis techniques to guide selection of critical program execution points in anomalous application runs. Our novel temporal ordering engine then leverages this information along with the application's static control structure to apply data flow analysis techniques to determine key application data such as loop control variables. We then use lightweight techniques to gather the dynamic data that determines the temporal order of the MPI tasks. Our evaluation, which extends the Stack Trace Analysis Tool (STAT), demonstrates that this temporal order analysis technique can isolate bugs in benchmark codes with injected faults as well as a real world hang case with AMG2006.

Abstract: Creating and debugging programs for wireless embedded networks (WENs) is notoriously difficult. Macroprogramming is an emerging technology that aims to address this problem by providing high-level programming abstractions. We present MDB, the first system to support the debugging of macroprograms. MDB allows the user to set break-points and step through a macroprogram using a source-level debugging interface similar to GDB, a process we call macrodebugging. A key challenge of MDB is to step through a macroprogram in sequential order even though it executes on the network in a distributed, asynchronous manner. Besides allowing the user to view distributed state, MDB also provides the ability to search for bugs over the entire history of distributed states. Finally, MDB allows the user to make hypothetical changes to a macroprogram and to see the effect on distributed state without the need to redeploy, execute, and test the new code. We show that macrodebugging is both easy and efficient: MDB consumes few system resources and requires few user commands to find the cause of bugs. We also provide a lightweight version of MDB called MDB Lite that can be used during the deployment phase to reduce resource consumption while still eliminating the possibility of heisenbugs: changes in the manifestation of bugs caused by enabling or disabling the debugger.

Abstract: The ability to dynamically change the behavior of an application is becoming an important issue in contemporary rich client software development. Not only can programmers benefit from dynamic updates during the development of concurrent applications where recreation of complex application states can be avoided during test and debugging but also at post-deployment time where applications can be updated transparently without going through the well-known halt, redeploy and restart scheme. In this paper, we explain how our dynamic update framework achieves transparent dynamic updates of running Java applications while guaranteeing both type and thread safety. A novel feature of our approach is that it supports full redefinition of classes by allowing changes to the type hierarchy. Our approach is based on a lightweight runtime system, which is injected into an application via bytecode transformations at class loading. We show how our approach can add dynamic update capabilities to rich client development by integrating it with the NetBeans rich client platform. Performance experiments on our NetBeans implementation show that the overhead of our approach is low when applied to component application programming interface classes. To the best of our knowledge no other existing approach achieves the same level of low overhead and programming transparency. Copyright © 2009 John Wiley & Sons, Ltd.

Abstract: Researchers continue to demonstrate the benefits of Mining Software Repositories (MSR) for supporting software development and research activities. However, as the mining process is time and resource intensive, they often create their own distributed platforms and use various optimizations to speed up and scale up their analysis. These platforms are project-specific, hard to reuse, and offer minimal debugging and deployment support. In this paper, we propose the use of MapReduce, a distributed computing platform, to support research in MSR. As a proof-of-concept, we migrate J-REX, an optimized evolutionary code extractor, to run on Hadoop, an open source implementation of MapReduce. Through a case study on the source control repositories of the Eclipse, BIRT and Datatools projects, we demonstrate that the migration effort to MapReduce is minimal and that the benefits are significant, as running time of the migrated J-REX is only 30% to 50% of the original J-REX's. This paper documents our experience with the migration, and highlights the benefits and challenges of the MapReduce framework in the MSR community.

Abstract: A system and method for debugging a computer program by using a call graph. A call graph that represents trace events during execution of a debuggee program may be used as input to a system that enables a user to debug the debuggee program. Mechanisms facilitate conditionally forming clusters of event nodes, a cluster indicative of multiple event nodes corresponding to an execution of a source language statement. During a debugging session, in response to a command to perform a step operation, the nodes of a cluster are processed together so that a step corresponds to multiple events if the multiple events correspond to a single source language statement. A mechanism for inspecting variables is provided. Variable values may be selectively propagated and provided based on the call graph and a static control flow analysis of the debuggee program.