Purpose

Valid defects, when fixed, improve the quality of the software, while, on the contrary, IN-valid defects waste team's time - need understand a defect, reproduce it, design and implement a fix, verify a fix. All that costs precious man-hours, not adding any value.   

The "Invalid Defects" metric shows a quality of defect submission process. It reflects a percentage of invalid defects (i.e. defects which cannot be reproduced, duplicated ones, or those rejected due to any reason) vs. all closed defects for a particular month.

How metric helps

Analyzing the validness of reported defects is very important since a significant number of invalid defects may be an indicator that the test team does not understand the product well enough, or doesn't pay enough attention while filling bug reports - all this leads to a waste of time and effort for other people in the team who are reviewing such bugs. A periodic analysis of this metric is an excellent indicator of how the test team is performing. 

Metric:

• shows a quality of defect submission process

• shows quality of testing

• shows how the QA team understands the product

• indicates if the team wastes too much time on valueless work

• shows if team is aware of the project QA processes 

• shows quality of defect life cycle 

• shows effectiveness and competency of a testing team

How metric works

Chart overview

Chart displays percentage of invalid defects (Axis Y) on a timeline (Axis X) where every bar is an exact ratio value for a particular month. There is also a trend line on the chart.

Open

Chart legend displays last calculated metric value and the trend in comparison with the previous time period.
By hover over a bar a hint appears showing a number of invalid defects ratio for a particular period.

TOP-5 problems metric identifies 

1. Lack of product knowledge in QA team

2. Poor requirements (Not ready for development backlog is taken into work, Poor requirements provisioning process)

3. There is no Test Plan and/or Strategy

4. Low quality of testing, i.e. poor test cases or not proper test techniques applied

5. Team misbalance (skills ratio)

Calculation 

Invalid Defects by Month, % = N_id / N_d * 100 %,

where 

N_id - number of defects closed in a considered month and satisfying both criteria:

1. Project Configuration > Data Sources > JIRA > Scope Management -> Definition of Work Buckets -> Defects

2. Project Configuration > Data Sources > JIRA > Quality Management > Criteria for invalid defects.

N_d - number of defects closed in a considered month and satisfying the criteria set in Project Configuration > Data Sources > Task Tracking System> Scope Management -> Definition of Work Buckets -> Defects

RAG thresholds: Red - metric value > 15 items; Amber - metric value <= 15  or > 10; Green - metric value <= 10.

Data Source

Data for the metric can be collected from a task tracking system (Jira, TFS, Rally, etc.).