Backlog Health

Purpose

Backlog Health shows the amount of a ready scope (i.e. scope in the "Ready for Development" state) at the start of each sprint. The scope may be measured in originally estimated hours or story points.

How metric helps

Backlog Health:

  • shows how many sprints of "ready for development" work a team possesses

  • helps to evaluate scope gathering and requirements management process

  • helps to understand whether the client/customer is able to provide enough details to form a backlog

  • shows how reachable Definition of Ready is, i.e. no issues in the estimation process, designs and specification are provided, INVEST principles easily followed

  • whether PO is engaged / available enough to manage backlog including priorities

  • shows whether the team is able to identify and define technical requirements to get tech tasks to a status of "ready for development"

  • shows whether there is an actual roadmap preserved and a clear product goal set to have backlog items in order

  • shows whether there is any risk that refined requirements can be subject to change and revisited moving forward

  • shows how effective feature refinement and estimation process is

Benefits:

  • make sure the business team provides enough requirements to backlog about feature development

  • make sure the engineering team conducts timely refinement and estimation of requirements

How metric works

Chart overview

The chart shows Backlog Health in a number of sprints scope is ready ahead - Axis Y by sprints in time - Axis X.

On hover over a column a hint appears with the following info: 

  • Sum of story points/original hours for the items in a ready status;

  • Number of items by type;

  • Average team velocity;

  • Metric value.

Chart legend shows the latest metric value and the difference between this value and the previous one.

By click on a column a pop up appears with the following information got from the task tracking system: 

  • Issue ID

  • Type

  • Priority

  • Summary.

Sprints on the widget are sorted by the sprint start date, as the metric is calculated at a sprint start.

Not estimated and issues with 0 estimate are not displayed in the pop up and not included in the calculation.

What metric means

  1. Absence of roadmap, no vision

  2. Poor requirements provisioning process

  3. Poor dependencies execution

  4. Scope cannot be properly estimated

  5. PO is not involved enough into Sprint execution

  • [Below ideal, <5] Features cannot be broken down to follow INVEST principles

  • [Below ideal, <5] Weak demand intake flow, the process when an idea gets into product backlog is not properly defined 

  • [Below ideal, <5] Multiple stakeholders influence priorities, leading the backlog mess

  • [Below ideal, <5] Rapid priority / roadmap changes 

  • [Below ideal, <5] There are mixed roles between SM, PO, BA in backlog management, so no unique approach in preparing and moving story to the definition of ready

  • [Below ideal, <5] Team does not possess enough technical background to specify technical requirements 

  • [Below ideal, <5] Team setup / composition misbalance (skills ratio)

  • [Above ideal, >5] Low velocity of the team, so the backlog is being burnt out slowly 

  • [Above ideal, >5] There are too many requests and ideas brainstormed in advance, and being provided as requirements, groomed, and estimated as user stories, however not planned far away

  • [Above ideal, >5] There are several streams/teams using the same backlog

Calculation

Backlog Health =∑Eready/Vavg,

where

∑Eready - estimates sum for N ready for development tasks at a sprint start. Ready state is according to criteria in Project settings>Scope management>Definition of Ready.

Vavg - average velocity for the last N periods; N periods  - taken from Project settings>Scope management> Calculation depth for average team velocity estimation.

RAG thresholds: Red - metric value <=1; Amber - 1 < metric value<=2; Green - 2 < metric value <=5; Amber - metric value >5.

Calculation notes

1. Issues that got a ready state before a considered sprint start timestamp are included into the calculation regardless of their assignment to another sprint.

2. Average velocity assumptions:

  • Incomplete (current) sprint is not taken into account;

  • Estimate is taken at an issue completion;

  • Not estimated and null estimates are not included in the calculation;

  • Sub-items are not included in the calculation.

3. Trend line (blue dotted line) calculation (its inclination to axis x) ignores an incomplete iteration in the chart to reveal an accurate tendency.

4. If there are several active sprints are running simultaneously in one project, the Backlog health value is calculated only for the first one.

Example

There are 2 sprints:

  • Current sprint S1: start on Dec 1.

  • Next sprint S2 : start on Dec 14.

There is an issue A that has got a ready state on Nov 30 but assigned to S2.

Issue A will be included in the calculation of Backlog Health for S1 and can be viewed in the drill-down for this sprint.

Data source 

Data for the metric can be collected from a task tracking system (Jira, TFS, Rally, etc.).

PerfQL

with all_sprints as (
select
id,
name,
coalesce(activated_date, start_date) + (sprint_starting_grace_period())::INTERVAL as start_date,
coalesce(complete_date, finish_date) as finish_date,
state,
row_number () over (order by start_date desc) as row_number
from sprint
where state != ''FUTURE'' and start_date is not null and start_date < now() and (finish_date is not null or complete_date is not null)
order by start_date desc
),

sprints as (
select * from all_sprints where row_number <= depth_for_avg_velocity_sprint() * 1 + 6 order by start_date
),

scope as (
select * from ticket
where parent_task is null and (done_date is null or done_date > (select min(start_date) from sprints))
),

-- Generate ticket snaphsots
snapshots as (
select * from ticket_snapshots(ARRAY(select key from scope))
),

snapshot_on_sprint_start as (
select distinct on (h.id, s.id) h.id as issue_id, s.id as sprint, s.start_date, h.snapshot_created
from snapshots h, sprints s
where h.snapshot_created < s.start_date
order by h.id, s.id, h.snapshot_created desc
),

snapshot_on_sprint_finish as (
select distinct on (h.id, s.id) h.id as issue_id, s.id as sprint, s.finish_date, h.snapshot_created
from snapshots h, sprints s
where h.snapshot_created < s.finish_date
order by h.id, s.id, h.snapshot_created desc
),

-- Generate ready by date
ready_on_sprint_start as(
select sum(sn.story_points) as story_points, sprint, array_agg(issue_id) as tickets from snapshots sn
inner join snapshot_on_sprint_start
on sn.id = snapshot_on_sprint_start.issue_id
and sn.snapshot_created = snapshot_on_sprint_start.snapshot_created
where is_ready(sn) and sn.story_points IS NOT NULL
group by sprint
),

completed_on_sprint_finish as (
select sum(sn.story_points) as story_points, sprint from snapshots sn
inner join snapshot_on_sprint_finish
on sn.id = snapshot_on_sprint_finish.issue_id
and sn.snapshot_created = snapshot_on_sprint_finish.snapshot_created
where sn.done_date < snapshot_on_sprint_finish.finish_date and sprint = any(sn.sprints)
group by sprint
),

average_velocity as (
select s.id as sprint, state, finish_date, start_date,
AVG(sum(coalesce(story_points,0))::numeric) OVER (ORDER BY s.finish_date asc ROWS BETWEEN (select depth_for_avg_velocity_sprint()-1) PRECEDING AND CURRENT ROW) AS average_velocity
from sprints s left join completed_on_sprint_finish on completed_on_sprint_finish.sprint = s.id
group by s.id, s.finish_date, state, start_date
),

backlog_health as (
select sprint,
case when s.state = ''CLOSED'' then lag(s.average_velocity) OVER (ORDER BY s.start_date asc)
when s.state = ''ACTIVE'' then (select average_velocity from average_velocity where state = ''CLOSED'' order by finish_date ASC limit 1) end
as avg_velocity
from average_velocity s)

select s.name as "Sprint",
(case
when avg_velocity = 0 then 0
when story_points is null then 0
else round((r.story_points/avg_velocity)::numeric,1)
end) as "Backlog Health"
from sprints s
left join ready_on_sprint_start r on s.id = r.sprint
inner join backlog_health v on v.sprint = s.id
where row_number <= 6
order by start_date

See alsoÂ