TL;DR: Aron is a freely available library dedicated to the static analysis of the numerical variables of programs by abstract interpretation, and its goal is to provide analysis implementers with ready-to-use numerical abstractions under a unified API.
Abstract: This article describes Apron , a freely available library dedicated to the static analysis of the numerical variables of programs by abstract interpretation. Its goal is threefold: provide analysis implementers with ready-to-use numerical abstractions under a unified API, encourage the research in numerical abstract domains by providing a platform for integration and comparison, and provide teaching and demonstration tools to disseminate knowledge on abstract interpretation.
TL;DR: Yogi, a tool that checks properties of C programs by combining static analysis and testing, is presented, which implements the Dash algorithm which performs verification by combining directed testing and abstraction.
Abstract: We present Yogi , a tool that checks properties of C programs by combining static analysis and testing Yogi implements the Dash algorithm which performs verification by combining directed testing and abstraction We have engineered Yogi in such a way that it plugs into Microsoft's Static Driver Verifier framework We have used this framework to run Yogi on 69 Windows Vista drivers with 85 properties We find that the new algorithm enables Yogi to scale much better than Slam , which is the current engine driving Microsoft's Static Driver Verifier
TL;DR: In this paper, a unified approach is used to handle the postbuckling response of composite beams with and without imperfection, and an analytical solution for the beam's static response in terms of the applied axial load, imperfection and lay up is obtained.
TL;DR: This paper presents a working prototype that can handle a variety of malware binaries, packed with both custom and commercial packers, and containing several examples of dynamic defenses.
Abstract: Current malware is often transmitted in packed or encrypted form to prevent examination by anti-virus software.To analyze new malware, researchers typically resort to dynamic code analysis techniques to unpack the code for examination.Unfortunately, these dynamic techniques are susceptible to a variety of anti-monitoring defenses, as well as "time bombs" or "logic bombs," and can be slow and tedious to identify and disable. This paper discusses an alternative approach that relies on static analysis techniques to automate this process. Alias analysis can be used to identify the existence of unpacking,static slicing can identify the unpacking code, and control flow analysis can be used to identify and neutralize dynamic defenses. The identified unpacking code can be instrumented and transformed, then executed to perform the unpacking.We present a working prototype that can handle a variety of malware binaries, packed with both custom and commercial packers, and containing several examples of dynamic defenses.
TL;DR: This work developed two novel binary analysis techniques: on-the-fly analysis of optimized machine code to enable minimally intrusive and accurate attribution of costs to dynamic calling contexts; and post-mortem analysis of optimizing machine code and its debugging sections to recover its program structure and reconstruct a mapping back to its source code.
Abstract: Modern programs frequently employ sophisticated modular designs. As a result, performance problems cannot be identified from costs attributed to routines in isolation; understanding code performance requires information about a routine's calling context. Existing performance tools fall short in this respect. Prior strategies for attributing context-sensitive performance at the source level either compromise measurement accuracy, remain too close to the binary, or require custom compilers. To understand the performance of fully optimized modular code, we developed two novel binary analysis techniques: 1) on-the-fly analysis of optimized machine code to enable minimally intrusive and accurate attribution of costs to dynamic calling contexts; and 2) post-mortem analysis of optimized machine code and its debugging sections to recover its program structure and reconstruct a mapping back to its source code. By combining the recovered static program structure with dynamic calling context information, we can accurately attribute performance metrics to calling contexts, procedures, loops, and inlined instances of procedures. We demonstrate that the fusion of this information provides unique insight into the performance of complex modular codes. This work is implemented in the HPCToolkit performance tools (http://hpctoolkit.org).
TL;DR: In this paper, the authors propose a typecheckable ownership domain annotations that enable a points-to static analysis to extract a sound global object graph that provides architectural abstraction by ownership hierarchy and by types where architecturally significant objects appear near the top of the hierarchy and data structures are further down.
Abstract: An object diagram makes explicit the object structures that are only implicit in a class diagram. An object diagram may be missing and must extracted from the code. Alternatively, an existing diagram may be inconsistent with the code, and must be analyzed for conformance with the implementation. One can generalize the global object diagram of a system into a runtime architecture which abstracts objects into components, represents how those components interact, and can decompose a component into a nested sub-architecture.A static object diagram represents all objects and inter-object relations possibly created, and is recovered by static analysis of a program. Existing analyses extract static object diagrams that are non-hierarchical, do not scale, and do not provide meaningful architectural abstraction. Indeed, architectural hierarchy is not readily observable in arbitrary code. Previous approaches used breaking language extensions to specify hierarchy and instances in code, or used dynamic analyses to extract dynamic object diagrams that show objects and relations for a few program runs.Typecheckable ownership domain annotations use existing language support for annotations and specify in code object encapsulation, logical containment and architectural tiers. These annotations enable a points-to static analysis to extract a sound global object graph that provides architectural abstraction by ownership hierarchy and by types, where architecturally significant objects appear near the top of the hierarchy and data structures are further down.Another analysis can abstract an object graph into a built runtime architecture. Then, a third analysis can compare the built architecture to a target, analyze and measure their structural conformance, establish traceability between the two and identify interesting differences.
TL;DR: In this article, the authors introduce a scheme for static analysis that allows us to partition large geometric datasets at multiple levels of granularity to achieve both load balancing in parallel computations and minimal access to secondary memory in out-of-core computations.
Abstract: In this paper we introduce a scheme for static analysis that allows us to partition large geometric datasets at multiple levels of granularity to achieve both load balancing in parallel computations and minimal access to secondary memory in out-of-core computations. The idea is illustrated and fully exploited for the case of isosurface extraction, but extendible to a class of algorithms based on a small set of parameters and for which an appropriate static analysis can be performed.
TL;DR: This evaluation, which extends the Stack Trace Analysis Tool (STAT), demonstrates that this temporal order analysis technique can isolate bugs in benchmark codes with injected faults as well as a real world hang case with AMG2006.
Abstract: We present a scalable temporal order analysis technique that supports debugging of large scale applications by classifying MPI tasks based on their logical program execution order. Our approach combines static analysis techniques with dynamic analysis to determine this temporal order scalably. It uses scalable stack trace analysis techniques to guide selection of critical program execution points in anomalous application runs. Our novel temporal ordering engine then leverages this information along with the application's static control structure to apply data flow analysis techniques to determine key application data such as loop control variables. We then use lightweight techniques to gather the dynamic data that determines the temporal order of the MPI tasks. Our evaluation, which extends the Stack Trace Analysis Tool (STAT), demonstrates that this temporal order analysis technique can isolate bugs in benchmark codes with injected faults as well as a real world hang case with AMG2006.
TL;DR: Tests with sample system confirm viability of proposed combination of static and live analysis, and investigator can have interactive session with virtual machine without violating evidence integrity.
Abstract: Traditional digital forensics is performed through static analysis of data preserved on permanent storage media. Not all data needed to understand the state of examined system exists in nonvolatile memory. Live analysis uses running system to obtain volatile data for deeper understanding of events going on. Sampling running system might irreversibly change its state making collected evidence invalid. This paper proposes combination of static and live analysis. Virtualization is used to bring static data to life. Volatile memory dump is used to enable offline analysis of live data. Using data from memory dump, virtual machine created from static data can be adjusted to provide better picture of the live system at the time when the dump was made. Investigator can have interactive session with virtual machine without violating evidence integrity. Tests with sample system confirm viability of proposed approach.
TL;DR: The SolidWorks Simulation 2009 software as discussed by the authors is a tool for finite element analysis (FEA) and it can be used to perform linear static analysis of parts and assemblies, as well as nonlinear analysis of nonlinear problems.
Abstract: Engineering Analysis with SolidWorks Simulation 2009 goes beyond the standard software manual because its unique approach concurrently introduces you to the SolidWorks Simulation 2009 software and the fundamentals of Finite Element Analysis (FEA) through hands-on exercises. A number of projects are presented using commonly used parts to illustrate the analysis features of SolidWorks Simulation. This book covers the following FEA functionality of SolidWorks Simulation 2009: Linear static analysis of parts and assemblies Frequency (modal) analysis Buckling analysis Thermal analysis Drop test analysis Optimization analysis Nonlinear analysis Dynamic analysis Table of Contents Before You Start 1. Introduction 2. Static analysis of a plate 3. Static analysis of an L-bracket 4. Stress and frequency analysis of a thin plate 5. Static analysis of a link 6. Frequency analysis of a tuning fork 7. Thermal analysis of a pipeline component and heater 8. Thermal analysis of a heat sink 9. Static analysis of a hanger 10. Analysis of contact stress between two plates 11. Thermal stress analysis of a bi-metal beam 12. Buckling analysis of an L-beam 13. Design optimization of a plate in tension 14. Static analysis of a bracket using adaptive solution methods 15. Design sensitivity analysis of hinge supported beam 16. Drop test of a porcelain ring 17. Selected nonlinear problems 18. Mixed meshing problem 19. Analysis of a weldment using beam elements 20. Dynamic Analysis - Modal Time History and Harmonic 21. Analysis of random vibration 22. Miscellaneous topics 23. Implementation of FEA into the design process 24. Glossary of terms 25. Resources available to FEA Users
TL;DR: A model to specify the local execution context of a basic block as a set of parameters can then be computed as a function of these parameters, which can be used for computing the Worst-Case Execution Time of the program.
Abstract: The static analysis of the execution time of a program (i.e. the evaluation of this time for any input data set) can be useful for the purpose of optimizing the code or verifying that strict real-time deadlines can be met. This analysis generally goes through determining the execution times of partial execution paths, typically basic blocks. Now, as soon as the target processor architecture features a superscalar pipeline, possibly with dynamic instruction scheduling, the execution time of a basic block highly depends on the pipeline state, that is on the instructions executed before it. In this paper, we propose a model to specify the local execution context of a basic block as a set of parameters. The execution time of the block can then be computed as a function of these parameters. We show how this model can be used to determine an upper bound of the execution time of a basic block, that can be used for computing the Worst-Case Execution Time of the program. Experimental results give an insight into the tightness of the estimations.
TL;DR: In this paper, a nonlinear dynamic response optimization of a joined wing is carried out by using equivalent static loads, which are the load sets that generate the same response field in linear static analysis as that in non-linear dynamic analysis.
Abstract: The joined-wing configuration that was published by Wolkovich in 1986 has been studied by many researchers (Wolkovich, J., "The Joined-Wing: An Overview," Journal of Aircraft, Vol. 23, No. 3, 1986, pp. 161―178. doi: 10.2514/3.45285). Thejoined-wing airplane is defined as an airplane that incorporates tandem wings arranged to form diamond shapes from both the top and front views. The joined wing can lead to increased aerodynamic performances as well as a reduction in the structural weight. However, the joined wing has high geometric nonlinearity under the gust load. The gust load acts as a dynamic load. Therefore, nonlinear dynamic (transient) behavior of the joined wing should be considered in structural optimization. In previous research, linear dynamic response optimization and nonlinear static response optimization were performed. It is well known that conventional nonlinear dynamic response optimization is extremely expensive. Therefore, in this research, nonlinear dynamic response optimization of a joined wing is carried out by using equivalent static loads. The concept of equivalent static loads is expanded and newly proposed for nonlinear dynamic response optimization. Equivalent static loads are the load sets that generate the same response field in linear static analysis as that in nonlinear dynamic analysis. Therefore, nonlinear dynamic response optimization can be conducted by repeated use of linear response optimization. For the verification of efficiency of the proposed method, a simple nonlinear dynamic response optimization problem is introduced. The problem is solved by using both the equivalent static loads method and the conventional method with sensitivity analysis using the finite difference method. The procedure for nonlinear dynamic response optimization of a joined wing using equivalent static loads is explained, and the optimum results are discussed.
TL;DR: An explicit parametrization of the static output feedback control gains that solve the mixed Hinfin and positive real performance problem and develops an explicit expression for calculating the H Infin norm of these systems.
TL;DR: A pen-based system that reconstructs 3D spatial geometry from a single 2D freehand-sketch consisting of straight and curved lines in interactive time and an iterative, Tablet-PC-based design system that uses the proposed reconstruction algorithm to recover 3D objects from 2D orthographic sketches.
Abstract: When designing a 3D object, designers, engineers and teachers often begin investigating potential design tradeoffs by creating informal sketches. Ideally, these sketches-in combination with a variety of engineering analysis tools-would allow prediction of the object's physical properties, especially those that affect the critical early design process. We introduce a pen-based system that reconstructs 3D spatial geometry from a single 2D freehand-sketch consisting of straight and curved lines in interactive time. Several optimization-based approaches to this problem have been proposed, but these generally have difficulty converging to an acceptable solution because the dimensionality of the search space is large. The primary contribution of this paper is a new reconstruction algorithm for orthographic projections of 3D wireframes. The algorithm reconstructs the depths of each vertex by exploiting geometric regularities among the graph lines in a reduced solution space, then optimizes a cost function over this space to recover the vertex depths. A second optimization algorithm is used to infer the 3D geometry of curved strokes once the vertex depths have been recovered. The proposed approach can recover the geometry of several objects with approximately 50 curved strokes in near interactive time. We also present an iterative, Tablet-PC-based design system that uses the proposed reconstruction algorithm to recover 3D objects from 2D orthographic sketches. The system allows the reconstructed objects to be subjected to two types of physical analysis, the results of which are superimposed directly on the sketch: a fast, kinematic simulation, and a complete finite-element-based static analysis. The object can quickly be modified in place using the pen-based interface according to the results of the analysis to allow for iterative design work. We demonstrate the system in action on a variety of early-stage design analyses.
TL;DR: In this article, the static bending, free vibration, and dynamic response of functionally graded piezoelectric panels have been carried out by finite element method under different sets of sets.
Abstract: In this article, analysis of the static bending, free vibration, and dynamic response of functionally graded piezoelectric panels have been carried out by finite element method under different sets...
TL;DR: A lemma is proved and can be used to select proper control parameters to guarantee the well-definedness of the controller and Experimental results indicate the proposed control system achieves 20% of the tracking error of a conventional PID control.
TL;DR: A new technique able to recover behavioral design pattern instances which combines static analysis, based on visual language parsing, with dynamic analysis,based on source code instrumentation is presented.
Abstract: In this paper we present a new technique able to recover behavioral design pattern instances which combines static analysis, based on visual language parsing, with dynamic analysis, based on source code instrumentation. In particular, the dynamic analysis is performed through the automatic instrumentation of the method calls involved in the candidate pattern instances identified during static analysis. The results obtained from a program monitoring activity are matched against the definitions of the pattern behaviors expressed in terms of monitoring grammars. We also present and discuss the results of a case study on JHotDraw 5.1 software library performed to assess the retrieval effectiveness of the proposed approach.
TL;DR: The majority of vulnerabilities that affect web applications can be ascribed to the lack of proper validation of user's input, before it is used as argument of an output function.
Abstract: Increasingly, web applications handle sensitive data and interface with critical back-end components, but are often written by poorly experienced programmers with low security skills. The majority of vulnerabilities that affect web applications can be ascribed to the lack of proper validation of user's input, before it is used as argument of an output function. Several program analysis techniques were proposed to automatically spot these vulnerabilities. One particularly effective is dynamic taint analysis. Unfortunately, this approach introduces a significant run-time penalty. In this paper, we present a hybrid analysis framework that blends together the strengths of static and dynamic approaches for the detection of vulnerabilities in web applications: a static analysis, performed just once, is used to reduce the run-time overhead of the dynamic monitoring phase. We designed and implemented a tool, called Phan, that is able to statically analyze PHP bytecode searching for dangerous code statements; then, only these statements are monitored during the dynamic analysis phase.
TL;DR: In this article, the authors combine static analysis, source code instrumentation and feedback-guided fuzz testing to automatically detect resource exhaustion denial of service attacks in software and generate inputs of coma for vulnerable code segments.
Abstract: Embodiments of the present invention combine static analysis, source code instrumentation and feedback-guided fuzz testing to automatically detect resource exhaustion denial of service attacks in software and generate inputs of coma for vulnerable code segments. The static analysis of the code highlights portions that are potentially vulnerable, such as loops and recursions whose exit conditions are dependent on user input. The code segments are dynamically instrumented to provide a feedback value at the end of each execution. Evolutionary techniques are then employed to search among the possible inputs to find inputs that maximize the feedback score.
TL;DR: This study shows that symbolic method is preferred for expressions with higher order cancelation and for programs without strong cancelation, any method works fairly well and GIA slightly outperforms others.
Abstract: Precision analysis and optimization is very important when transforming a floating-point algorithm into fixed-point hardware implementations. The core analysis techniques are either based on dynamic analysis or static analysis. We believe in static error analysis, as it is the only technique that can guarantee the desired worst-case accuracy. In this paper we study various underlying arithmetic candidates that can be used in static error analysis and compare their computed sensitivities. The approaches studied include Affine Arithmetic(AA), General Interval Arithmetic (GIA) and Automatic Differentiation (Symbolic Arithmetic). Our study shows that symbolic method is preferred for expressions with higher order cancelation. For programs without strong cancelation, any method works fairly well and GIA slightly outperforms others. We also study the impact of program transformations on these arithmetics.
TL;DR: This paper shows that semantic context information such as modes enhances performance analysis and prediction by ruling out infeasible worstcase situations that lead to overly conservative performance predictions.
Abstract: Bosch has established Component Based Software Development (CBSD) for automotive systems, which are resource constrained real-time embedded systems such as engine control systems. Classical CBSD approaches enable effective software reuse mainly in functional aspects by managing complexity with abstraction and encapsulation. However, to fully exploit the advantages of CBSD for real-time embedded systems, non-functional system properties such as timing and memory usage need to be addressed by the underlying component model. It is important that non-functional properties have a certain degree of precision to ensure hardware dimensioning and cost optimization for such systems. Static analysis methods used to extract or analyze nonfunctional properties (e.g., worst case execution time) in most cases introduce overestimation which is a hindrance for accurate prediction of non-functional properties. Therefore, accurate prediction of system properties requires specifying semantic context information such as modes in the component model to reduce overestimation. This paper describes how we extend the Bosch software component model to specify non-functional component properties with modes information. We demonstrate how mode dependent timing behavior is automatically extracted from the software, specified in the component specification and used for analysis and prediction in real-time embedded systems. This paper shows that semantic context information such as modes enhances performance analysis and prediction by ruling out infeasible worstcase situations that lead to overly conservative performance predictions.
TL;DR: The a posteriori criterion can be applied to FE problems of linear static analysis or thermal problems for stationary linear conduction and is able to estimate the influence of shape transformations over the global analysis results.
TL;DR: This work introduces a framework of test case comparison metrics which will quantitatively describe the distance between any arbitrary test case pair of an existing test suite, allowing various test case analysis applications.
Abstract: Selection of diverse test cases and elimination of duplicates are two major problems in product testing life cycle, especially in sustained engineering environment In order to solve these, we introduce a framework of test case comparison metrics which will quantitatively describe the distance between any arbitrary test case pair of an existing test suite, allowing various test case analysis applications We combine program profiles from test execution, static analysis and statistical techniques to capture various aspects of test execution and compute a specialized test case distance measurement Using these distance metrics, we drive a customized hierarchical test suite clustering algorithm that groups similar test cases together We present an industrial strength framework called SPIRiT that works at binary level, implementing different metrics in the form of coverage, control, data, def-use, temporal variances and does test case clustering This is step towards integrating runtime analysis, static analysis, statistical techniques and machine learning to drive new generation of test suite analysis algorithms
TL;DR: Some relationships between software analysis and software quality characters are introduced, so as to provide some hints when some specific software character is under analyzing.
Abstract: Research on software analysis has long history.It has been widely used in many processes in software lifecycle.The software analysis technologies that are used in different processes are different,while there are many interleaves among them.This paper discusses the concept of software analysis,followed with main software analysis technologies and related tools,from view of static analysis and dynamic analysis.Some relationships between software analysis and software quality characters are introduced,so as to provide some hints when some specific software character is under analyzing.The future of software analysis is discussed in the end of this paper.
TL;DR: This paper decomposes the vulnerability of a register into intrinsic and conditional basic-block vulnerabilities, which allows for a fast, yet reasonably accurate, linear equation-based RF vulnerability estimation mechanism.
Abstract: With continuous technology scaling, soft errors are becoming an increasingly important design concern even for earth-bound applications. While compiler approaches have the potential to mitigate the effect of soft errors with minimal runtime overheads, static vulnerability estimation---an essential part of compiler approaches---is lacking due to its inherent complexity. This paper presents a static analysis approach for Register File (RF) vulnerability estimation. We decompose the vulnerability of a register into intrinsic and conditional basic-block vulnerabilities. This decomposition allows us to develop a fast, yet reasonably accurate, linear equation-based RF vulnerability estimation mechanism. We demonstrate its practical application to compiler optimizations. Our experimental results on benchmarks from MiBench suite indicate that not only our static RF vulnerability estimation is fast and accurate, but also compiler optimizations enabled by our static estimation can achieve very cost-effective protection of register files against soft errors.
TL;DR: A technique that exploits the notion of footprints and memoization to compute individual abstract transformers more efficiently is introduced and used to prove properties of fine-grained concurrent programs with a shared, mutable, heap in the presence of an unbounded number of objects and threads.
Abstract: We present a new technique for speeding up static analysis of (shared memory) concurrent programs. We focus on analyses that compute thread correlations : such analyses infer invariants that capture correlations between the local states of different threads (as well as the global state). Such invariants are required for verifying many natural properties of concurrent programs.
Tracking correlations between different thread states, however, is very expensive. A significant factor that makes such analysis expensive is the cost of applying abstract transformers. In this paper, we introduce a technique that exploits the notion of footprints and memoization to compute individual abstract transformers more efficiently.
We have implemented this technique in our concurrent shape analysis framework. We have used this implementation to prove properties of fine-grained concurrent programs with a shared, mutable, heap in the presence of an unbounded number of objects and threads. The properties we verified include memory safety, data structure invariants, partial correctness, and linearizability. Our empirical evaluation shows that our new technique reduces the analysis time significantly (e.g., by a factor of 35 in one case).
TL;DR: In this paper, a numerical procedure for the static analysis of arch-supported tensile structures with block and tackle suspension system is presented, which is suitable for a structural analysis both in the prestressing process and in the final state under external loads.
Abstract: A numerical procedure for the static analysis of arch-supported tensile structures with block and tackle suspension system is presented. The procedure, based on dynamic relaxation, is suitable for a structural analysis both in the prestressing process and in a final state under external loads. The friction between the pulley and its shaft is also taken into account in the analysis. After the introduction of the developed procedure, two structures are presented as examples. The analysis of a very simple structure validates the procedure, then the analysis of a more complex structure, an arch-supported cable net roof illustrates the stability and efficiency of the procedure.
TL;DR: In this article, a mixed least-squares finite element model was developed for static and free vibration analysis of laminated composite plates, where the first-order shear deformation theory with generalized displacements and stress resultants were considered as independent variables, using equal and high-order C 0 basis functions.
TL;DR: This work presents a novel semi-static approach for resolving dynamic class loading by combining static string analysis with dynamically gathered information about the execution environment, and proposes extensions of string analysis to increase the number of sites that can be resolved purely statically, and to track the names of environment variables.
Abstract: In Java software, one important flexibility mechanism is dynamic class loading. Unfortunately, the vast majority of static analyses for Java treat dynamic class loading either unsoundly or too conservatively. We present a novel semi-static approach for resolving dynamic class loading by combining static string analysis with dynamically gathered information about the execution environment. The insight behind the approach is that dynamic class loading often depends on characteristics of the environment that are encoded in various environment variables. Such variables are not static elements; however, their run-time values typically remain the same across multiple executions of the application. Thus, the string values reported by our technique are tailored to the current installation of the system under analysis. Additionally, we propose extensions of string analysis to increase the number of sites that can be resolved purely statically, and to track the names of environment variables. An experimental evaluation on the Java 1.4 standard libraries shows that a state-of-the-art purely static approach resolves only 28% of non-trivial sites, while our approach resolves 74% of such sites. We also demonstrate how the information gained from resolved dynamic class loading can be used to determine the classes that can potentially be instantiated through the use of reflection. Our extensions of string analysis greatly increase the number of resolvable reflective instantiation sites. This work is a step towards making static analysis tools better equipped to handle the dynamic features of Java.
TL;DR: The NIST SAMATE project conducted the first Static Analysis Tool Exposition (SATE) in 2008 to advance research in static analysis tools that find security defects in source code and identifies several ways in which the released data and analysis are useful.
Abstract: The NIST SAMATE project conducted the first Static Analysis Tool Exposition (SATE) in 2008 to advance research in static analysis tools that find security defects in source code. The main goals of SATE were to enable empirical research based on large test sets and to encourage improvement and speed adoption of tools. The exposition was planned to be an annual event. Briefly, participating tool makers ran their tool on a set of programs. Researchers led by NIST performed a partial analysis of tool reports. The results and experiences were reported at the Static Analysis Workshop in Tucson, AZ, in June, 2008. The tool reports and analysis were made publicly available in 2009. This paper describes the SATE procedure, provides our observations based on the data collected, and critiques the exposition, including the lessons learned that may help future expositions. This paper also identifies several ways in which the released data and analysis are useful. First, the output from running many tools on production software can be used for empirical research. Second, the analysis of tool reports indicates weaknesses that exist in the software and that are reported by the tools. Finally, the analysis may also be used as a building block for a further study of the weaknesses and of static analysis.