Abstract: In this paper we present Mayhem, a new system for automatically finding exploitable bugs in binary (i.e., executable) programs. Every bug reported by Mayhem is accompanied by a working shell-spawning exploit. The working exploits ensure soundness and that each bug report is security-critical and actionable. Mayhem works on raw binary code without debugging information. To make exploit generation possible at the binary-level, Mayhem addresses two major technical challenges: actively managing execution paths without exhausting memory, and reasoning about symbolic memory indices, where a load or a store address depends on user input. To this end, we propose two novel techniques: 1) hybrid symbolic execution for combining online and offline (concolic) execution to maximize the benefits of both techniques, and 2) index-based memory modeling, a technique that allows Mayhem to efficiently reason about symbolic memory at the binary level. We used Mayhem to find and demonstrate 29 exploitable vulnerabilities in both Linux and Windows programs, 2 of which were previously undocumented.

Abstract: Networks are complex and prone to bugs. Existing tools that check configuration files and data-plane state operate offline at timescales of seconds to hours, and cannot detect or prevent bugs as they arise.Is it possible to check network-wide invariants in real time, as the network state evolves? The key challenge here is to achieve extremely low latency during the checks so that network performance is not affected. In this paper, we present a preliminary design, VeriFlow, which suggests that this goal is achievable. VeriFlow is a layer between a software-defined networking controller and network devices that checks for network-wide invariant violations dynamically as each forwarding rule is inserted. Based on an implementation using a Mininet OpenFlow network and Route Views trace data, we find that VeriFlow can perform rigorous checking within hundreds of microseconds per rule insertion.

Abstract: The behavior of a Software-Defined Network is controlled by programs, which like all software, will have bugs - but this programmatic control also enables new ways to debug networks. This paper introduces ndb, a prototype network debugger inspired by gdb, which implements two primitives useful for debugging an SDN: breakpoints and packet backtraces. We show how ndb modifies forwarding state and logs packet digests to rebuild the sequence of events leading to an errant packet, providing SDN programmers and operators with a valuable tool for tracking down the root cause of a bug.

Abstract: Testing and debugging is the most expensive, error-prone phase in the software development life cycle. Automated testing and diagnosis of software faults can drastically improve the efficiency of this phase, this way improving the overall quality of the software. In this paper we present a toolset for automatic testing and fault localization, dubbed GZoltar, which hosts techniques for (regression) test suite minimization and automatic fault diagnosis (namely, spectrum-based fault localization). The toolset provides the infrastructure to automatically instrument the source code of software programs to produce runtime data. Subsequently the data was analyzed to both minimize the test suite and return a ranked list of diagnosis candidates. The toolset is a plug-and-play plug-in for the Eclipse IDE to ease world-wide adoption.

Abstract: Given limited resource and time before software release, development-site testing and debugging become more and more insufficient to ensure satisfactory software performance. As a counterpart for debugging in the large pioneered by the Microsoft Windows Error Reporting (WER) system focusing on crashing/hanging bugs, performance debugging in the large has emerged thanks to available infrastructure support to collect execution traces with performance issues from a huge number of users at the deployment sites. However, performance debugging against these numerous and complex traces remains a significant challenge for performance analysts. In this paper, to enable performance debugging in the large in practice, we propose a novel approach, called StackMine, that mines callstack traces to help performance analysts effectively discover highly impactful performance bugs (e.g., bugs impacting many users with long response delay). As a successful technology-transfer effort, since December 2010, StackMine has been applied in performance-debugging activities at a Microsoft team for performance analysis, especially for a large number of execution traces. Based on real-adoption experiences of StackMine in practice, we conducted an evaluation of StackMine on performance debugging in the large for Microsoft Windows 7. We also conducted another evaluation on a third-party application. The results highlight substantial benefits offered by StackMine in performance debugging in the large for large-scale software systems.

Abstract: Diagnosing software failures in the field is notoriously difficult, in part due to the fundamental complexity of troubleshooting any complex software system, but further exacerbated by the paucity of information that is typically available in the production setting. Indeed, for reasons of both overhead and privacy, it is common that only the run-time log generated by a system (e.g., syslog) can be shared with the developers. Unfortunately, the ad-hoc nature of such reports are frequently insufficient for detailed failure diagnosis. This paper seeks to improve this situation within the rubric of existing practice. We describe a tool, LogEnhancer that automatically “enhances” existing logging code to aid in future post-failure debugging. We evaluate LogEnhancer on eight large, real-world applications and demonstrate that it can dramatically reduce the set of potential root failure causes that must be considered while imposing negligible overheads.

Abstract: Effective debugging is crucial to producing dependable software. Manual debugging is becoming prohibitively expensive, especially due to the growing size and complexity of programs. Given that fault localization is one of the most expensive activities in program debugging, there has been a great demand for fault localization techniques that can help guide programmers to the locations of faults. In this paper a technique named DStar (D*), which has its origins rooted in similarity coefficient-based analysis, is proposed, which can identify suspicious locations for fault localization automatically without requiring any prior information on program structure or semantics. D* is evaluated across 21 programs and is compared to 16 different fault localization techniques. Both single-fault and multi-fault programs are used. Results indicate that D* is more effective at locating faults than all the other techniques it is compared to.

Abstract: Runtime monitoring is a technique usable in all phases of the software development cycle, from initial testing, to debugging, to actually maintaining proper function in production code. Of particular importance are parametric monitoring systems, which allow the specification of properties that relate objects in a program, rather than only global properties. In the past decade, a number of parametric runtime monitoring systems have been developed. Here we give a demonstration of our system, JavaMOP. It is the only parametric monitoring system that allows multiple differing logical formalisms. It is also the most efficient in terms of runtime overhead, and very competitive with respect to memory usage.

Abstract: We present MLE+, a tool for energy-efficient building automation design, co-simulation and analysis. The tool leverages the high-fidelity building simulation capabilities of EnergyPlus and the scientific computation and design capabilities of Matlab for controller design. MLE+ facilitates integrated building simulation and controller formulation with integrated support for system identification, control design, optimization, simulation analysis and communication between software applications and building equipment. It provides streamlined workflows, a graphical front-end, and debugging support to help control engineers eliminate design and programming errors and take informed decisions early in the design stage, leading to fewer iterations in the building automation development cycle. We show through an example and two case studies how MLE+ can be used for designing energy-efficient control algorithms for both simulated buildings in EnergyPlus and real building equipment via BACnet.

Abstract: While users today have access to many tools that assist in performing large scale data analysis tasks, understanding the performance characteristics of their parallel computations, such as MapReduce jobs, remains difficult. We present PerfXplain, a system that enables users to ask questions about the relative performances (i.e., runtimes) of pairs of MapReduce jobs. PerfXplain provides a new query language for articulating performance queries and an algorithm for generating explanations from a log of past MapReduce job executions. We formally define the notion of an explanation together with three metrics, relevance, precision, and generality, that measure explanation quality. We present the explanation-generation algorithm based on techniques related to decision-tree building. We evaluate the approach on a log of past executions on Amazon EC2, and show that our approach can generate quality explanations, outperforming two naive explanation-generation methods.

Abstract: Reverse debugging is the ability of a debugger to stop after a failure in a program has been observed and go back into the history of the execution to uncover the reason for the failure. Long the dream of programmers, over the past decade, reverse execution has become a practical technique available in a number of free and commercial tools. This article will review the history and techniques of reverse debugging, as researched, implemented, and used from the 1970s until today. We will provide some personal insights into reverse debugging, from our own practical use of one such tool, Wind River Simics.

Abstract: Research on engaging young children in computer programming to develop high-level cognitive skills has suggested that debugging is among the most important actions leading to the development of logical thinking, problem solving, and social interaction skills. Although there have been a significant amount of studies done in this area, the debugging tools and techniques have been developed only as models and instructional methodologies outside of the tool itself. This work presents the design and analysis of debugging abilities embedded into a tangible programming system called Robo-Blocks. Students create a program by connecting physical command blocks, which then wirelessly controls the motion of a floor robot. Debugging is achieved by allowing children to run their program in a step-by-step manner and use passive objects to recognize and identify problems.Our evaluation with 52 children ages 8-9 has shown that (1) although tangible programming has the benefit of being exceptionally engaging to young children, early primary school children can quickly loose attention when no progress is made on a particular problem unless there are heuristics provided to help them move forward (2) Robo-Block's framework supplements the existing instructional methodologies used in the debugging process. Students showed significant increase in the ability to analyze problems and think of ways to correct them.

Abstract: During the process of developing and maintaining a complex software system, developers pose detailed questions about the runtime behavior of the system. Source code views offer strictly limited insights, so developers often turn to tools like debuggers to inspect and interact with the running system. Unfortunately, traditional debuggers focus on the runtime stack as the key abstraction to support debugging operations, though the questions developers pose often have more to do with objects and their interactions. We propose object-centric debugging as an alternative approach to interacting with a running software system. We show how, by focusing on objects as the key abstraction, natural debugging operations can be defined to answer developer questions related to runtime behavior. We present a running prototype of an object-centric debugger, and demonstrate, with the help of a series of examples, how object-centric debugging offers more effective support for many typical developer tasks than a traditional stack-oriented debugger.

Abstract: A computer-implemented method, apparatus and computer program product for debugging programs, the method comprising: displaying a graphic waveform showing values of one or more state variables of a computer program being debugged in two or more points in time; receiving a user selection from points in time, of an indication to a selected point in time in execution from the graphic waveform; and resuming within a debugger an execution state of the computer program associated with the selected point in time.

Abstract: Introducing partial programmability in circuits by replacing some gates with look up tables (LUTs) can be an effective way to improve post-silicon or in-field rectification and debugging Although finding configurations of LUTs that can correct the circuits can be formulated as a QBF problem, solving it by state-of-the-art QBF solvers is still a hard problem for large circuits and many LUTs In this paper, we present a rectification and debugging method for combinational circuits with LUTs by repeatedly applying Boolean SAT solvers Through the experimental results, we show our proposed method can quickly find LUT configurations for large circuits with many LUTs, which cannot be solved by a QBF solver

Abstract: The invention discloses a semi-physical simulation testing platform of a train control network, which comprises a vehicle-mounted electrical control system, a software simulation control system and/or a physical control system and a data acquisition interactive system, wherein the software simulation control system and/or the physical control system is connected with the vehicle-mounted electrical control system; the data acquisition interactive system is connected with the vehicle-mounted electrical control system so as to convert a signal output by the vehicle-mounted electrical control system into a digital signal or convert the digital signal into an input signal required by the vehicle-mounted electrical control system; and the data acquisition interactive system is connected with the software simulation control system and/or the physical control system so as to convert the digital signal connected and output by the software simulation control system and/or the physical control system into a switching value signal or convert the switching value signal into the digital signal required by the software simulation control system and/or the physical control system. The semi-physical simulation testing platform of the train control network is capable of comprehensively implementing evaluation and verification on the design of a train network control system, eliminating possibledesign errors, rapidly positioning faults to ask for reasons, and reducing the production debugging cost and the technical risks of real trains.

Abstract: Integrated Development Environments (IDE) generate multiple graphical and textual representations of programs. Co-ordination of these representations during program comprehension and debugging can be a complex task. In order to better understand the role and effectiveness of multiple representations, we conducted an empirical study of Java program debugging with a professional, multi-representation IDE. We found that program code and dynamic representations (dynamic viewer, variable watch and output) attracted the most attention of programmers. Static representations like Unified Modeling Language (UML) Diagrams and Control Structure Diagrams (CSD) saw significantly lesser usage. We analyzed gaze patterns by segmenting the debugging sessions into three, five and fifteen minute intervals, and classifying gazes into short and long gazes. Novel data mining techniques were used to detect high frequency patterns from eye tracking data. Visual pattern differences were found among participants based on their programming experience, familiarity with the IDE and debugging performance.

Abstract: Modern software systems must support a high degree of variability to accommodate a wide range of requirements and operating conditions. This paper introduces the Abstract Behavioural Specification (ABS) language and tool suite, a comprehensive platform for developing and analysing highly adaptable distributed concurrent software systems. The ABS language has a hybrid functional and object- oriented core, and comes with extensions that support the development of systems that are adaptable to diversified requirements, yet capable to maintain a high level of trustworthiness. Using ABS, system variability is consistently traceable from the level of requirements engineering down to object behaviour. This facilitates temporal evolution, as changes to the required set of features of a system are automatically reflected by functional adaptation of the system's behaviour. The analysis capabilities of ABS stretch from debugging, observing and simulating to resource analysis of ABS models and help ensure that a system will remain dependable throughout its evolutionary lifetime. We report on the experience of using the ABS language and the ABS tool suite in an industrial case study.

Abstract: A system and method are provided for enhanced navigation along execution time and code space in a debugger to assist a user in remediating errors, streamlining, or reverse engineering a computer program and the source code thereof. Snapshots of system states are recorded, a causality tree of commands is constructed through execution of the program to be debugged, and an intelligent display of system states captured during runtime and indexed or cross-referenced by time are displayed to the user in an intelligent manner to aid the user with certain debugging tasks. Additionally, further features in assisting the user to locate a root cause of an error or unexpected value and remediate that cause are also provided.

Abstract: We aim to detect and diagnose code misbehavior that wastes energy, which we call energy bugs. This paper describes a method and implementation, called Carat, for performing such diagnosis on mobile devices. Carat takes a collaborative, black-box approach. A non-invasive client app sends intermittent, coarse-grained measurements to a server, which identifies correlations between higher expected energy use and client properties like the running apps, device model, and operating system. Carat successfully detected all energy bugs in a controlled experiment and, during a deployment to 883 users, identified 5434 instances of apps exhibiting buggy behavior in the wild.

Abstract: To develop a model of students' debugging processes, I conducted a qualitative analysis of young students engaged in debugging computer programs they had written in the programming language Scratch. I present a microgenetic analysis that tracks how one student's attention to elements of computer program state shifted during his debugging process. I present evidence that this student had relevant domain knowledge and claim that his changing attention within the problem, and not his domain knowledge, mediated his debugging process. I hypothesize that a key competence in debugging is learning to identify what elements of program state are important to pay attention to and that this attention, and not only domain knowledge, mediates the debugging process. This hypothesis is consistent with a model of physics reasoning and learning from the Knowledge in Pieces theoretical framework and in this research I build upon education research outside of computer science. The case study analyzes the debugging process of a student entering the sixth grade, but I document an isomorphic case from a pair of college students to show that this pattern extends beyond this age.

Abstract: Learn to build fast and scalable software in JavaScript with Node.jsNode.js is a powerful and popular new framework for writing scalable network programs using JavaScript. This no nonsense book begins with an overview of Node.js and then quickly dives into the code, core concepts, and APIs. In-depth coverage pares down the essentials to cover debugging, unit testing, and flow control so that you can start building and testing your own modules right away.Covers node and asynchronous programming main conceptsAddresses the basics: modules, buffers, events, and timersExplores streams, file systems, networking, and automated unit testingGoes beyond the basics, and shares techniques and tools for debugging, unit testing, and flow control If you already know JavaScript and are curious about the power of Node.js, then this is the ideal book for you.

Abstract: Change impact analysis techniques estimate the potential effects of changes made to software Directed Incremental Symbolic Execution (DiSE) is an intraprocedural technique for characterizing the impact of software changes on program behaviors DiSE first estimates the impact of the changes on the source code using program slicing techniques, and then uses the impact sets to guide symbolic execution to generate path conditions that characterize impacted program behaviors DiSE, however, cannot reason about the flow of impact between methods and will fail to generate path conditions for certain impacted program behaviors In this work, we present iDiSE, an extension to DiSE that performs an interprocedural analysis iDiSE combines static and dynamic calling context information to efficiently generate impacted program behaviors across calling contexts Information about impacted program behaviors is useful for testing, verification, and debugging of evolving programs We present a case-study of our implementation of the iDiSE algorithm to demonstrate its efficiency at computing impacted program behaviors Traditional notions of coverage are insufficient for characterizing the testing efforts used to validate evolving program behaviors because they do not take into account the impact of changes to the code In this work we present novel definitions of impacted coverage metrics that are useful for evaluating the testing effort required to test evolving programs We then describe how the notions of impacted coverage can be used to configure techniques such as DiSE and iDiSE in order to support regression testing related tasks We also discuss how DiSE and iDiSE can be configured for debugging; finding the root cause of errors introduced by changes made to the code In our empirical evaluation we demonstrate that the configurations of DiSE and iDiSE can be used to support various software maintenance tasks

Abstract: As more and more multitier services are developed from commercial off-the-shelf components or heterogeneous middleware without source code available, both developers and administrators need a request tracing tool to (1) exactly know how a user request of interest travels through services of black boxes and (2) obtain macrolevel user request behaviors of services without manually analyzing massive logs. This need is further exacerbated by IT system “agility,” which mandates the tracing tool to provide online performance data since offline approaches cannot reflect system changes in real time. Moreover, considering the large scale of deployed services, a pragmatic tracing approach should be scalable in terms of the cost in collecting and analyzing logs. In this paper, we introduce a precise, scalable, and online request tracing tool for multitier services of black boxes. Our contributions are threefold. First, we propose a precise request tracing algorithm for multitier services of black boxes, which only uses application-independent knowledge. Second, we present a microlevel abstraction, component activity graph, to represent causal paths of each request. On the basis of this abstraction, we use dominated causal path patterns to represent repeatedly executed causal paths that account for significant fractions, and we further present a derived performance metric of causal path patterns, latency percentages of components, to enable debugging performance-in-the-large. Third, we develop two mechanisms, tracing on demand and sampling, to significantly increase the system scalability. We implement a prototype of the proposed system, called PreciseTracer, and release it as open source code. In comparison with WAP5-a black-box tracing approach, PreciseTracer achieves higher tracing accuracy and faster response time. Our experimental results also show that PreciseTracer has low overhead, and still achieves high tracing accuracy even if an aggressive sampling policy is adopted, indicating that PreciseTracer is a promising tracing tool for large-scale production systems.