Abstract: The reality of multi-core hardware has made concurrent programs pervasive. Unfortunately, writing correct concurrent programs is difficult. Addressing this challenge requires advances in multiple directions, including concurrency bug detection, concurrent program testing, concurrent programming model design, etc. Designing effective techniques in all these directions will significantly benefit from a deep understanding of real world concurrency bug characteristics.This paper provides the first (to the best of our knowledge) comprehensive real world concurrency bug characteristic study. Specifically, we have carefully examined concurrency bug patterns, manifestation, and fix strategies of 105 randomly selected real world concurrency bugs from 4 representative server and client open-source applications (MySQL, Apache, Mozilla and OpenOffice). Our study reveals several interesting findings and provides useful guidance for concurrency bug detection, testing, and concurrent programming language design.Some of our findings are as follows: (1) Around one third of the examined non-deadlock concurrency bugs are caused by violation to programmers' order intentions, which may not be easily expressed via synchronization primitives like locks and transactional memories; (2) Around 34% of the examined non-deadlock concurrency bugs involve multiple variables, which are not well addressed by existing bug detection tools; (3) About 92% of the examined concurrency bugs canbe reliably triggered by enforcing certain orders among no more than 4 memory accesses. This indicates that testing concurrent programs can target at exploring possible orders among every small groups of memory accesses, instead of among all memory accesses; (4) About 73% of the examinednon-deadlock concurrency bugs were not fixed by simply adding or changing locks, and many of the fixes were not correct at the first try, indicating the difficulty of reasoning concurrent execution by programmers.

Abstract: Concurrency is pervasive in large systems. Unexpected interference among threads often results in "Heisenbugs" that are extremely difficult to reproduce and eliminate. We have implemented a tool called CHESS for finding and reproducing such bugs. When attached to a program, CHESS takes control of thread scheduling and uses efficient search techniques to drive the program through possible thread interleavings. This systematic exploration of program behavior enables CHESS to quickly uncover bugs that might otherwise have remained hidden for a long time. For each bug, CHESS consistently reproduces an erroneous execution manifesting the bug, thereby making it significantly easier to debug the problem. CHESS scales to large concurrent programs and has found numerous bugs in existing systems that had been tested extensively prior to being tested by CHESS. CHESS has been integrated into the test frameworks of many code bases inside Microsoft and is used by testers on a daily basis.

Abstract: Execution replay of virtual machines is a technique which has many important applications, including debugging, fault-tolerance, and security. Execution replay for single processor virtual machines is well-understood, and available commercially. With the advancement of multi-core architectures, however, multiprocessor virtual machines are becoming more important. Our system, SMP-ReVirt, is the first system to log and replay a multiprocessor virtual machine on commodity hardware. We use hardware page protection to detect and accurately replay sharing between virtual cpus of a multi-cpu virtual machine, allowing us to replay the entire operating system and all applications. We have tested our system on a variety of workloads, and find that although sharing under SMP-ReVirt is expensive, for many workloads and applications, including debugging, the overhead is acceptable.

Abstract: When software developers want to understand the reason for a program's behavior, they must translate their questions about the behavior into a series of questions about code, speculating about the causes in the process. The Whyline is a new kind of debugging tool that avoids such speculation by instead enabling developers to select a question about program output from a set of why did and why didn't questions derived from the program's code and execution. The tool then finds one or more possible explanations for the output in question, using a combination of static and dynamic slicing, precise call graphs, and new algorithms for determining potential sources of values and explanations for why a line of code was not reached. Evaluations of the tool on one task showed that novice programmers with the Whyline were twice as fast as expert programmers without it. The tool has the potential to simplify debugging in many software development contexts.

Abstract: This paper reviews the literature related to the learning and teaching of debugging computer programs. Debugging is an important skill that continues to be both difficult for novice programmers to learn and challenging for computer science educators to teach. These challenges persist despite a wealth of important research on the subject dating back as far as the mid 1970s. Although the tools and languages novices use for writing programs today are notably different from those employed decades earlier, the basic problem-solving and pragmatic skills necessary to debug them effectively are largely similar. Hence, an understanding of the previous work on debugging can offer computer science educators insights into how to improve contemporary learning and teaching of debugging and may suggest directions for future research into this important area. This overview of the debugging literature is organized around four questions relevant to computer science educators and education researchers: What causes bugs to o...

Abstract: Debugging is often difficult and frustrating for novices. Yet because students typically debug outside the classroom and often in isolation, instructors rarely have the opportunity to closely observe students while they debug. This paper describes the details of an exploratory study of the debugging skills and behaviors of contemporary novice Java programmers. Based on a modified replication of Katz and Anderson's study of novices, we sought to broadly survey the modern landscape of novice debugging abilities. As such, this study reports general quantitative results and fills in the picture with qualitative detail from a relatively small, but varied sample. Comprehensive interviews involving both a programming and a debugging task, followed by a semi-structured interview and a questionnaire, were conducted with 21 CS2 students at seven colleges and universities. While many subjects successfully debugged a representative set of typical CS1 bugs, there was a great deal of variation in their success at the p...

Abstract: Multiprocessor deterministic replay has many potential uses in the era of multicore computing, including enhanced debugging, fault tolerance, and intrusion detection. While sources of nondeterminism in a uniprocessor can be recorded efficiently in software, it seems likely that hardware support will be needed in a multiprocessor environment where the outcome of memory races must also be recorded.We develop a memory race recording mechanism, called Rerun, that uses small hardware state (~166 bytes/core), writes a small race log (~4 bytes/kilo- instruction), and operates well as the number of cores per system scales (e.g., to16cores). Rerun exploits the dual of conventional wisdom in race recording: Rather than record information about individual memory accesses that conflict, we record how long a thread executes without conflicting with other threads. In particular, Rerun passively creates atomic episodes. Each episode is a dynamic instruction sequence that a thread happens to execute without interacting with other threads. Rerun uses Lamport Clocks to order episodes and enable replay of an equivalent execution.

Abstract: Various implementation approaches for developing a domain-specific language are available in literature. There are certain common beliefs about the advantages/disadvantages of these approaches. However, it is hard to be objective and speak in favor of a particular one, since these implementation approaches are normally compared over diverse application domains. The purpose of this paper is to provide empirical results from ten diverse implementation approaches for domain-specific languages, but conducted using the same representative language. Comparison shows that these discussed approaches differ in terms of the effort need to implement them, however, the effort needed by a programmer to implement a domain-specific language should not be the only factor taken into consideration. Another important factor is the effort needed by an end-user to rapidly write correct programs using the produced domain-specific language. Therefore, this paper also provides empirical results on end-user productivity, which is measured as the lines of code needed to express a domain-specific program, similarity to the original notation, and how error-reporting and debugging are supported in a given implementation.

Abstract: Software vendors collect bug reports from customers to improve the quality of their software. These reports should include the inputs that make the software fail, to enable vendors to reproduce the bug. However, vendors rarely include these inputs in reports because they may contain private user data. We describe a solution to this problem that provides software vendors with new input values that satisfy the conditions required to make the software follow the same execution path until it fails, but are otherwise unrelated with the original inputs. These new inputs allow vendors to reproduce the bug while revealing less private information than existing approaches. Additionally, we provide a mechanism to measure the amount of information revealed in an error report. This mechanism allows users to perform informed decisions on whether or not to submit reports. We implemented a prototype of our solution and evaluated it with real errors in real programs. The results show that we can produce error reports that allow software vendors to reproduce bugs while revealing almost no private information.

Abstract: Software development for wireless sensor networks is a challenging and time consuming task. The resource limited hardware with limited I/O and debugging abilities combined with the often cumbersome hardware debugging tools makes debugging on the target hardware difficult. We present MSPsim, an extensible sensor board platform and MSP430 instruction level simulator. MSPsim is intended to be used for reducing development and debugging time by allowing low-level and fine grained instrumentation of various aspects of software execution. The use of a simulator also enables development and testing without access to the target hardware.

Abstract: Silicon validation - proving a chip works correctly at speed and in system under different operating conditions - is always necessary, even for a "perfect" design. Silicon debug - finding the root cause of a malfunction - is necessary whenever a design is not flawless. First-silicon validation and debug require a labor-intensive engineering effort of several months and have become the least predictable and most time-consuming part of a new 90-nm chip's development cycle. Lack of adequate tools and automatic procedures is a big factor in this bottleneck. The difficulty of silicon validation will increase at 65 nm and below because existing ad hoc methodologies don't scale with the unprecedented levels of SoC device complexity. Even the most sophisticated SoC design methodology cannot fully account for all the parameters that affect silicon behavior, or for all logic corner cases occurring in the life of a chip working at speed and in system. For example, the simultaneous occurrence of two unlikely events might not be anticipated pre- silicon, so it is never simulated or analyzed; however, when it occurs in system, it can cause unexpected behavior. Presilicon verification methods - simulation, emulation, FPGA prototyping, timing analysis, and formal verification - don't address many deep-submicron problems that occur in the actual device.

Abstract: A qualitative analysis of debugging strategies of novice Java programmers is presented. The study involved 21 CS2 students from seven universities in the U.S. and U.K. Subjects "warmed up" by coding a solution to a typical introductory problem. This was followed by an exercise debugging a syntactically correct version with logic errors. Many novices found and fixed bugs using strategies such as tracing, commenting out code, diagnostic print statements and methodical testing. Some competently used online resources and debuggers. Students also used pattern matching to detect errors in code that "just didn't look right". However, some used few strategies, applied them ineffectively, or engaged in other unproductive behaviors. This led to poor performance, frustration for some, and occasionally the introduction of new bugs. Pedagogical implications and suggestions for future research are discussed.

Abstract: This paper presents a tool for uncovering bugs due to interactive complexity in networked sensing applications. Such bugs are not localized to one component that is faulty, but rather result from complex and unexpected interactions between multiple often individually non-faulty components. Moreover, the manifestations of these bugs are often not repeatable, making them particularly hard to find, as the particular sequence of events that invokes the bug may not be easy to reconstruct. Because of the distributed nature of failure scenarios, our tool looks for sequences of events that may be responsible for faulty behavior, as opposed to localized bugs such as a bad pointer in a module. An extensible framework is developed where a front-end collects runtime data logs of the system being debugged and an offline back-end uses frequent discriminative pattern mining to uncover likely causes of failure. We provide a case study of debugging a recent multichannel MAC protocol that was found to exhibit corner cases of poor performance (worse than single channel MAC). The tool helped uncover event sequences that lead to a highly degraded mode of operation. Fixing the problem significantly improved the performance of the protocol.We also provide a detailed analysis of tool overhead in terms of memory requirements and impact on the running application.

Abstract: Answer-set programming (ASP) is widely recognised as a viable tool for declarative problem solving. However, there is currently a lack of tools for developing answer-set programs. In particular, providing tools for debugging answer-set programs has recently been identified as a crucial prerequisite for a wider acceptance of ASP. In this paper, we introduce a meta-programming technique for debugging in ASP. The basic question we address is why interpretations expected to be answer sets are not answer sets of the program to debug. We thus deal with finding semantical errors of programs. The explanations provided by our method are based on an intuitive scheme of errors that relies on a recent characterisation of the answer-set semantics. Furthermore, as we are using a meta-programming technique, debugging queries are expressed in terms of answer-set programs themselves, which has several benefits: For one, we can directly use ASP solvers for processing debugging queries. Indeed, our technique can easily be implemented, and we devised a corresponding prototype debugging system. Also, our approach respects the declarative nature of ASP, and the capabilities of the system can easily be extended to incorporate differing debugging features.

Abstract: Coping with software defects that occur in the post-deployment stage is a challenging problem: bugs may occur only when the system uses a specific configuration and only under certain usage scenarios. Nevertheless, halting production systems until the bug is tracked and fixed is often impossible. Thus, developers have to try to reproduce the bug in laboratory conditions. Often the reproduction of the bug consists of the lion share of the debugging effort. In this paper we suggest an approach to address the aforementioned problem by using a specialized runtime environment (QVM, for Quality Virtual Machine). QVM efficiently detects defects by continuously monitoring the execution of the application in a production setting. QVM enables the efficient checking of violations of user-specified correctness properties, e.g., typestate safety properties, Java assertions, and heap properties pertaining to ownership. QVM is markedly different from existing techniques for continuous monitoring by using a novel overhead manager which enforces a user-specified overhead budget for quality checks. Existing tools for error detection in the field usually disrupt the operation of the deployed system. QVM, on the other hand, provides a balanced trade off between the cost of the monitoring process and the maintenance of sufficient accuracy for detecting defects. Specifically, the overhead cost of using QVM instead of a standard JVM, is low enough to be acceptable in production environments. We implemented QVM on top of IBM's J9 Java Virtual Machine and used it to detect and fix various errors in real-world applications.

Abstract: Back-in-time debuggers are extremely useful tools for identifying the causes of bugs. Unfortunately the "omniscient" approaches that try to remember allprevious states are impractical because they consume too much space or they are far too slow. Several approaches rely on heuristics to limit these penalties, but they ultimately end up throwing out too much relevant information. In this paper we propose a practical approach that attempts to keep track of only the relevant data. In contrast to other approaches, we keep object history information together with the regular objects in the application memory. Although seemingly counter-intuitive, this approach has the effect that data not reachable from current application objects (and hence, no longer relevant) is garbage collected. We describe the technical details of our approach, and we present benchmarks that demonstrate that memory consumption stays within practical bounds. Furthermore, the performance penalty is significantly less than with other approaches.

Abstract: Execution indexing uniquely identifies a point in an execution. Desirable execution indices reveal correlations between points in an execution and establish correspondence between points across multiple executions. Therefore, execution indexing is essential for a wide variety of dynamic program analyses, for example, it can be used to organize program profiles; it can precisely identify the point in a re-execution that corresponds to a given point in an original execution and thus facilitate debugging or dynamic instrumentation. In this paper, we formally define the concept of execution index and propose an indexing scheme based on execution structure and program state. We present a highly optimized online implementation of the technique. We also perform a client study, which targets producing a failure inducing schedule for a data race by verifying the two alternative happens-before orderings of a racing pair. Indexing is used to precisely locate corresponding points across multiple executions in the presence of non-determinism so that no heavyweight tracing/replay system is needed.

Abstract: Techniques for debugging a computer program that includes multiple modules executing on multiple machines include receiving, at a unifying component, first data from a first machine. The first data indicates debugging information generated by the first machine. Second data is also received at the unifying component from a second machine. The second data indicates debugging information generated by the second machine. Based on the first data and the second data, third data is formed indicating a single integrated representation of debugging information for the computer program. The unifying component allows debugging information from several machines to be integrated and then presented to a user through a single debugger client.

Abstract: Multiple target devices, such as wireless phones, may receive configuration data, such as software updates, configuration settings, developer environment settings, etc., from a host computer that automatically detects characteristics of respective target devices and selects the appropriate configuration data. A configuration process may generally include identifying the target devices, determining characteristics of the target devices, such as device type, device storage capacity, operating system, operating system version, installed software, etc., and selecting configuration data for updating the target devices for communication with the host computer. Configuration data may include data that: allows the target device to establish an appropriate development environment with a developer computing device, such as settings and/or software for debugging, performance analysis, and/or software execution monitoring; indicates changes to the configuration and/or registry settings of a target device; configures, adds, updates, or removes software; updates operating system components; and/or personalizes and/or adds user interface components, for example.

Abstract: Most existing fault-localization techniques focus on identifying and reporting single statements that may contain a fault. Even in cases where a fault involves a single statement, it is generally hard to understand the fault by looking at that statement in isolation. Faults typically manifest themselves in a specific context, and knowing that context is necessary to diagnose and correct the fault. In this paper, we present a novel fault-localization technique that identifies sequences of statements that lead to a failure. The technique works by analyzing partial execution traces corresponding to failing executions and identifying common segments in these traces, incrementally. Our approach provides developers a context that is likely to result in a more directed approach to fault understanding and a lower overall cost for debugging.

Abstract: Assertion-based design is a powerful new paradigm that is facilitating quality improvement in electronic design. Assertions are statements used to describe properties of the design (I.e., design intent), that can be included to actively check correctness throughout the design cycle and even the lifecycle of the product. With the appearance of two new languages, PSL and SVA, assertions have already started to improve verification quality and productivity. This is the first book that presents an under-the-hood view of generating assertion checkers, and as such provides a unique and consistent perspective on employing assertions in major areas, such as: specification, verification, debugging, on-line monitoring and design quality improvement.

Abstract: Ontology mapping is the bottleneck in solving interoperation between Semantic Web applications using heterogeneous ontologies. Many mapping methods have been proposed in recent years, but in practice, it is still difficult to obtain satisfactory mapping results having high precision and recall. Different from existing methods, which focus on finding efficient and effective solutions for the ontology mapping problem, we place emphasis on analyzing the mapping result to detect/diagnose the mapping defects. In this paper, a novel technique called debugging ontology mappings is presented. During debugging, some types of mapping errors, such as redundant and inconsistent mappings, can be detected. Some warnings, including imprecise mappings or abnormal mappings, are also locked by analyzing the features of mapping result. More importantly, some errors and warnings can be repaired automatically or can be presented to users with revising suggestions. The experimental results reveal that the ontology debugging technique is promising, and it can improve the quality of mapping result.

Abstract: When developing and researching new trusted computing technologies, appropriate tools to investigate their behavior and to evaluate their performance are of paramount importance. In this paper, we present an efficient and portable TPM emulator for Unix. Our emulator enables not only the implementation of flexible and low-cost test-beds and simulators but, in addition, provides programmers of trusted systems with a powerful testing and debugging tool that can also be used for educational purposes. Thanks to its portability and interoperability, the TPM emulator runs on a variety of platforms and is compatible with the most relevant software packages and interfaces.

Abstract: The challenges in empirical eye-tracking studies of usability or complex problem solving include 1) how to effectively analyze the eye-tracking data, and 2) how to interpret and relate the resulting measures to the user cognitive processing. We conducted a reanalysis of eye-tracking data from a recent study that involved programmers of two experience groups debugging a program with the help of multiple representations. The proportional fixation time on each area of interest (AOI), frequency of visual attention switches between the areas, and the type of switch were investigated during five consequential phases of ten minutes of debugging. We increased the granularity of the focus on the user processing several times, allowing us to construct a better picture of the process. In addition, plotting the areas of interest in time supported a visual analysis and comparison with the quantitative data.We found repetitive patterns of visual attention that were associated with less experience in programming and lower performance. We also discovered that at the beginning of the process programmers made use of both the code and visualization while frequently switching between them. At a later stage of debugging, more experienced programmers began to increasingly integrate also the output of the program and employed a high-frequency of visual attention switching to coordinate the three representations.

Abstract: Due to the increasing complexity of today's circuits a high degree of automation in the design process is mandatory. The detection of faults and design errors is supported quite well using simulation or formal verification. But locating the fault site is typically a time consuming manual task. Techniques to automate debugging and diagnosis have been proposed. Approaches based on Boolean Satisfiability (SAT) have been demonstrated to be very effective. In this work debugging on the gate level is considered. Unsatisfiable cores contained in a SAT instance for debugging are used (1) to determine all suspects, and (2) to speed-up the debugging process. In comparison to standard SAT-based debugging, the experimental results show a significant speed-up for debugging multiple faults.

Abstract: While termination checking tailored to real-world library code or frameworks has received ever-increasing attention during the last years, the complementary question of disproving termination properties as a means of debugging has largely been ignored so far. We present an approach to automatic non-termination checking that relates to termination checking in the same way as symbolic testing does to program verification. Our method is based on the automated generation of invariants that show that terminating states of a program are unreachable from certain initial states. Such initial states are identified using constraint-solving techniques. The method is fully implemented on top of a program verification system and available for download. We give an empirical evaluation of the approach using a collection of non-terminating example programs.

Abstract: In this paper we present PROSE, a system that performs reversible and systematic changes to running Java applications without requiring them to be shut down. PROSE is motivated by scenarios such as hotfixes, online program instrumentation and debugging, and evolution of critical legacy applications. In PROSE, changes to running applications are performed by replacing method bodies. To select which code to replace, PROSE supports matching based on both type information and regular expressions. New code can invoke the method it replaces, facilitating code evolution. Changes are composable, and may be reordered or selectively withdrawn at any time. Furthermore, the dynamic changes are expressed as Java classes rather than through an additional programming language. We describe the architecture of PROSE, the challenges of using aggressive inlining to achieve performance, and use standard benchmarks to demonstrate code performance comparable with, or better than, compile-time systems from the Aspect-Oriented Programming community.

Abstract: Domain-specific languages (DSLs) assist a software developer (or end-user) in writing a program using idioms that are similar to the abstractions found in a specific problem domain. Tool support for DSLs is lacking when compared with the capabilities provided for standard general-purpose languages (GPLs), such as Java and C++. For example, support for debugging a program written in a DSL is often non-existent. The lack of a debugger at the proper abstraction level limits an end-user's ability to discover and locate faults in a DSL program. This paper describes a grammar-driven technique to build a debugging tool generation framework from existing DSL grammars. The DSL grammars are used to generate the hooks needed to interface with a supporting infrastructure constructed for an integrated development environment that assists in debugging a program written in a DSL. The contribution represents a coordinated approach to bring essential software tools (e.g. debuggers) to different types of DSLs (e.g. imperative, declarative, and hybrid). This approach hides from the end-users the accidental complexities associated with expanding the focus of a language environment to include debuggers. The research described in this paper addresses a long-term goal of empowering end-users with development tools for particular DSL problem domains at the proper level of abstraction without depending on a specific GPL. Copyright © 2007 John Wiley & Sons, Ltd.