Defect Density
- Natallia_Mendzel@epam.com
- Anastasia Baranouskaya
- Yuliya_Varonina
Purpose
Defect Density is the ratio of defects per software "size". Although there are known industry averages about typical defect densities per different programming languages and software frameworks, everyone would agree the less defects in the software the higher its quality. So, measuring it is the first step to improve the quality.
Defect Density metric in PERF shows a ratio of defects to a software size. PERF provides several views of this metric:
Defect Density per Story Point
Defect Density per non-Defect Item
Production Defect Density per release
How metric helps
Defect Density helps to understand a relative "level of quality" of the software. Being, by nature, normalized against the size of the delivered software, this metric can allow a comparison of different modules or even projects between each other. So, among other benefits it helps prioritize investments into quality improvements.
In general, Defect Density can help with -
identify the areas for improvement (a piece with the biggest defect density requires more attention and resources for quality improvements)
compare quality of components/software modules
shows the ratio of defects logged by internal team per delivered items (Ndefects/Nitems)
shows quality of application
shows quality of testing
identifies what area should be treated with higher attention
shows how effective changes that were done (if any) in QA process
How metric works
Chart overview
Chart shows a Defect Density (Axis Y) over months (Axis X).
On hover over a column a hint appears with the following info:
Month - a calendar month a metric is calculated for;
Defects submitted - a number of defects both internal and external submitted in a considered month excluding defects satisfying criteria in Project Settings>Task Tracking System>Quality Management>Criteria for escaped defects;
SP delivered (w/o defects) - software size in Story Points excluding defects in a considered month;
ORNon-defects accomplished (w/o defects) - software size in items excluding defects in a considered month;
Defect density - metric value (rounded to 2 decimal places).
On hover over a column a hint appears with the following info:
Month - a calendar month a metric is calculated for;
External defects - a number of external submitted in a considered month satisfying criteria in Project Settings>Task Tracking System>Quality Management>Criteria for escaped defects;
Affected versions - number of released versions in a considered month.
Defect density - metric value (rounded to 2 decimal places).
Chart legend shows the following:
the last calculated metric value;
a difference with the previous month.
Top problems metric identifies
Calculation
Defect Density per SP | Defect Density per item | Defect Density per release |
|---|---|---|
Defect Density per SP = (Ndefects - Ninvalid)/SPitems - SPdefects closed, where Ndefects - a number of created defects in a considered month; Ninvalid - a number of invalid defects in a considered month in any status; SPitems - estimate in SP for all completed items in a considered month; SPdefects closed - estimate in SP for all closed defects in a considered month. Sub-items are not included into the calculation. RAG thresholds: Red - metric value >= 3; Amber - 1 <= metric value < 3; Green - metric value < 1. | Defect Density per Item = ( Ndefects - Ninvalid)/Nitems - Ndefects closed, where Ndefects - a number of created defects in a considered month; Ninvalid - a number of invalid defects in a considered month in any status; Nitems - all completed items in a considered month; Ndefects closed - all closed defects in a considered month. Sub-items are not included into the calculation. RAG thresholds: Red - metric value >= 3; Amber - 1 <= metric value < 3; Green - metric value < 1. | Defect Density per release = Ntop external defects /Nreleases, where Ntop external defects - a number of external defects of a top priority (any status) in the versions released in a considered month, i.e. affected version a defect is assigned is released in this month. Top priority defects are defined in Project Settings>Task Tracking System>Quality Management>Top-priority defects; Nreleases- number of released versions in a considered month. Sub-items are not included into the calculation. RAG thresholds: no. |
Data Source
Data for the metric can be collected from a task tracking system (Jira, TFS, Rally, etc.).