DEFECT DENSITY METRICS

Size: px

Start display at page:

Download "DEFECT DENSITY METRICS"

Madeleine Thomas
6 years ago
Views:

1 1 DEFECT DENSITY METRICS

2 Defect density is a measure of the total known defects divided by the size of the software entity being measured. Estimated because defects are found throughout the entire life-cycle of the product Important for cost and resource estimates for the maintenance phase of the life cycle 2

3 The Number of Known Defects is the count of total defects identified against a particular software entity, during a particular time period. Examples include: The period might be for one of the following: for a duration (say, the first month, the quarter, or the year). Defect to date since the creation of module Defects found in a program during an inspection Defects to date since the shipment of a release to the customer for each phase of the software life cycle. for the whole of the software life cycle 3

4 Size is a normalize that allows comparisons between different software entities (i.e., modules, releases, products). Size is typically counted either in Lines of Code (LOC) or Function Points. 4

5 DEFECT DENSITY USES Defect Density is used to compare the relative number of defects in various software components. This helps identify candidates for additional inspection or testing or for possible reengineering or replacement. Identifying defect components allows the concentration of limited resources into areas with the highest potential return on the investment. 5

6 DEFECT DENSITY USES Another use for Defect Density is to compare subsequent releases of a product to track the impact of defect reduction and quality improvement activities. 6

7 DEFECT DENSITY ISSUES (1) There is no general consensus on: - What constitutes a defect - How to measure SW size (2) Defect Density is a product measure but it is derived from the process of finding defects. - May say more about quality of defect finding and reporting, than about the quality of the product itself (3) Should measure number of defects with respect to time (e.g.mtbf) rather than product size 7

8 DEFECT DENSITY CAUTIONS Even if we knew the exact number of residual faults in a system, it is still: - Difficult to determine in advance the seriousness of a fault - There is much variability in the way the system is used by different users; a system is not always used as expected 8

9 More specific Defect Density (rate) = number of defects / opportunities for errors during a specified time Number of defects can be approximated as equal to the number of unique causes of observed failures Opportunities for error can be expressed as KLOC Time frame (life of product or LOP) varies 9

10 Defect Density Example Product is one year old, and is 10 KLOC Unique causes of observed failures = 50 Current Defect Density = 50/10K = 5 defects per KLOC per year After second year Unique causes of observed failures = 75 Current Defect Density = 75 / 10K = 7.5 defects per KLOC per 2year or 3.75 per KLOC per year 10

11 Timeframes must be the same Cannot compare (current) defect rate for a one year old product to the (current) defect rate of a four year old product IBM considers life of product to be 4 years Must account for new and modified code in LOC count (otherwise metric is skewed) LOC counting must remain consistent 11

12 Measuring LOC Comparison Issues How LOC is calculated Count only executable lines Count executable lines, plus data definitions Count executable lines, plus data definitions, plus comments Count physical lines Count logical lines (terminated by ; ) Normalising quality/productivity metrics considering size Size usually LOC, KLOC, or Pages of Documentation Defects per KLOC $ per LOC Pages of documentation per KLOC LOC per person-month $ per page of documentation 12

13 At IBM, the following LOC count metrics, based on logical LOC, are used: (1) Shipped Source Instructions (SSI): LOC count for the total product (2) Changed Source Instructions (CSI): LOC count for the new and changed code of the new release. The relationship between SSI and CSI is given by: SSI (current release) = SSI (previous release) + CSI (new and changed code instructions for current release) - deleted code (usually very small) - changed code (to avoid double count in both SSI & CSI) Defects can be field defects (found by customers), or internal defects (found internally).

14 We are about to launch the third release of an inventory management software system. - The size of the initial release was 50 KLOC, with a defect density of 2.0 def/kssi - In the second release, 20KLOC of code was newly added or modified (20% of which are changed lines of code, with no deleted code ). - In the third release, 30KLOC of code was newly added or modified (20% of which are changed lines of code, with no deleted code ). Assume that the LOC counts are based on source instructions (SSI/CSI). (a) Considering that the quality goal of the company has been to achieve 10% improvement in overall defect rates from release-torelease, calculate the total number of additional defects for the third release. (b) What should be the maximum (overall) defect rate target for the third release to ensure that the number of new defects does not exceed that of the second release.

Defect Removal Metrics. SE 350 Software Process & Product Quality 1

Defect Removal Metrics. SE 350 Software Process & Product Quality 1 Defect Removal Metrics 1 Objectives Understand some basic defect metrics and the concepts behind them Defect density metrics Defect detection and removal effectiveness etc. Look at the uses and limitations