Purpose

Valid defects, when fixed, improve the quality of the software, while, on the contrary, IN-valid defects waste team's time - need understand a defect, reproduce it, design and implement a fix, verify a fix. All that costs precious man-hours, not adding any value.

The "Invalid Defects" metric shows a quality of defect submission process. It reflects a percentage of invalid defects (i.e. defects which cannot be reproduced, duplicated ones, or those rejected due to any reason) vs. all closed defects for a particular month.

How metric helps

Analyzing the validness of reported defects is very important since a significant number of invalid defects may be an indicator that the test team does not understand the product well enough, or doesn't pay enough attention while filling bug reports - all this leads to a waste of time and effort for other people in the team who are reviewing such bugs. A periodic analysis of this metric is an excellent indicator of how the test team is performing.

Metric:

shows a quality of defect submission process
shows quality of testing
shows how the QA team understands the product
indicates if the team wastes too much time on valueless work
shows if team is aware of the project QA processes
shows quality of defect life cycle
shows effectiveness and competency of a testing team

How metric works

Chart overview

Chart displays percentage of invalid defects (Axis Y) on a timeline (Axis X) where every bar is an exact ratio value for a particular month. There is also a trend line on the chart.

Chart legend displays last calculated metric value and the trend in comparison with the previous time period.
By hover over a bar a hint appears showing a number of invalid defects ratio for a particular period.

TOP-5 problems metric identifies

Lack of product knowledge in QA team
Poor requirements (Not ready for development backlog is taken into work, Poor requirements provisioning process)
There is no Test Plan and/or Strategy
Low quality of testing, i.e. poor test cases or not proper test techniques applied
Team misbalance (skills ratio)

QA processes are not established well
Absence or poor micro-change management process
Negative trends with other metrics e.g. Committed vs Completed, Average velocity by sprints, etc
Regression testing is executed too late on time

Calculation

Invalid Defects by Month, % = N_id / N_d * 100 %,

where

N_id - number of defects closed in a considered month and satisfying both criteria:

Project Configuration > Data Sources > JIRA > Scope Management -> Definition of Work Buckets -> Defects
Project Configuration > Data Sources > JIRA > Quality Management > Criteria for invalid defects.

N_d - number of defects closed in a considered month and satisfying the criteria set in Project Configuration > Data Sources > Task Tracking System> Scope Management -> Definition of Work Buckets -> Defects

RAG thresholds: Red - metric value > 15 items; Amber - metric value <= 15 or > 10; Green - metric value <= 10.

Data Source

Data for the metric can be collected from a task tracking system (Jira, TFS, Rally, etc.).