TL;DR: This work investigated whether software metrics obtained from source code and development history are discriminative and predictive of vulnerable code locations, and predicted over 80 percent of the known vulnerable files with less than 25 percent false positives for both projects.
Abstract: Security inspection and testing require experts in security who think like an attacker. Security experts need to know code locations on which to focus their testing and inspection efforts. Since vulnerabilities are rare occurrences, locating vulnerable code locations can be a challenging task. We investigated whether software metrics obtained from source code and development history are discriminative and predictive of vulnerable code locations. If so, security experts can use this prediction to prioritize security inspection and testing efforts. The metrics we investigated fall into three categories: complexity, code churn, and developer activity metrics. We performed two empirical case studies on large, widely used open-source projects: the Mozilla Firefox web browser and the Red Hat Enterprise Linux kernel. The results indicate that 24 of the 28 metrics collected are discriminative of vulnerabilities for both projects. The models using all three types of metrics together predicted over 80 percent of the known vulnerable files with less than 25 percent false positives for both projects. Compared to a random selection of files for inspection and testing, these models would have reduced the number of files and the number of lines of code to inspect or test by over 71 and 28 percent, respectively, for both projects.
TL;DR: This work proposes a novel mechanism - called detection strategy - for formulating metrics-based rules that capture deviations from good design principles and heuristics, and defined such detection strategies for capturing around ten important flaws of object-oriented design found in the literature.
Abstract: In order to support the maintenance of an object-oriented software system, the quality of its design must be evaluated using adequate quantification means. In spite of the current extensive use of metrics, if used in isolation metrics are oftentimes too fine grained to quantify comprehensively an investigated design aspect (e.g., distribution of system's intelligence among classes). To help developers and maintainers detect and localize design problems in a system, we propose a novel mechanism - called detection strategy - for formulating metrics-based rules that capture deviations from good design principles and heuristics. Using detection strategies an engineer can directly localize classes or methods affected by a particular design flaw (e.g., God Class), rather than having to infer the real design problem from a large set of abnormal metric values. We have defined such detection strategies for capturing around ten important flaws of object-oriented design found in the literature and validated the approach experimentally on multiple large-scale case-studies.
TL;DR: An analysis of the distributions and relationships derived from the change data collected during development of a medium-scale software project produces some surprising insights into the factors influencing software development.
Abstract: An analysis of the distributions and relationships derived from the change data collected during development of a medium-scale software project produces some surprising insights into the factors influencing software development. Among these are the tradeoffs between modifying an existing module as opposed to creating a new one, and the relationship between module size and error proneness.
TL;DR: This paper proposes to use network analysis on dependency graphs of the entire system to identify central program units that are more likely to face defects and finds that the recall for models building from network measures is by 10% points higher than for models built from complexity metrics.
Abstract: In software development, resources for quality assurance are limited by time and by cost. In order to allocate resources effectively, managers need to rely on their experience backed by code complexity metrics. But often dependencies exist between various pieces of code over which managers may have little knowledge. These dependencies can be construed as a low level graph of the entire system. In this paper, we propose to use network analysis on these dependency graphs. This allows managers to identify central program units that are more likely to face defects. In our evaluation on Windows Server 2003, we found that the recall for models built from network measures is by 10% points higher than for models built from complexity metrics. In addition, network measures could identify 60% of the binaries that the Windows developers considered as critical-twice as many as identified by complexity metrics.
TL;DR: This paper defines a measurement structure model that identifies the elementary component of measures and the measurement process, and considers five other models involved in measurement: unit definition models, instrumentation models, attribute relationship models, measurement protocols and entity population models.
Abstract: In this paper we propose a framework for validating software measurement. We start by defining a measurement structure model that identifies the elementary component of measures and the measurement process, and then consider five other models involved in measurement: unit definition models, instrumentation models, attribute relationship models, measurement protocols and entity population models. We consider a number of measures from the viewpoint of our measurement validation framework and identify a number of shortcomings; in particular we identify a number of problems with the construction of function points. We also compare our view of measurement validation with ideas presented by other researchers and identify a number of areas of disagreement. Finally, we suggest several rules that practitioners and researchers can use to avoid measurement problems, including the use of measurement vectors rather than artificially contrived scalars.