What Should Every Team Measure Using Allstacks?

Allstacks believes that all team should measure the following metrics, broken into the following themes:

  • Completion: Roadmap Focus, Velocity, Sprint Attainment

  • Efficiency: Cycle Time, Lead Time, Issue Time in State, Coding Days, PR Response Time

  • Quality: Currently Open Defects by Priority, Defects Created Over Time

  • Readiness: Refined Backlog

Note: All of these metrics can be up-leveled in visualizations for engineering directors and VPs. Please reach out to your Allstacks representative with any questions.

 

 

Completion:

Roadmap Focus

 

Why:

  • Understand how much of your team’s effort is going towards roadmap work vs maintenance and tasks.

  • Answer the question “How long do you think this will take?” with confidence knowing how focused you’ve historically been on planned work.

Goal:

  • Varies based on the team. True support teams often do very little roadmap work, but new product development teams should have a much higher ratio. If you see the ratio of uncategorized work increasing, meet with your team to determine what’s causing the distraction. Common issues include mounting bugs or tech-debt, or interrupting tasks ripe for automation.

Starting Metric:

  • Velocity, by issue count or story points.

Settings to Apply:

Velocity

 

Why:

  • Understand if your team is consistently delivering based on their estimation and pointing.

Goal:

  • Consistency proportional to team size.

Starting Metric:

  • Velocity

    • Save a view by issue count (to avoid hiding unestimated items) and story points.

Settings to Apply:

  • Use a default view utilizing story points or issue count based on how your team estimates work.

  • Filter out “Won’t Do” type statuses using advanced filters.

  • Group by month to avoid the inconsistencies of weeks.

Sprint Attainment (for Scrum Teams)

Why:

  • Understand if your team is correctly committing to planned work.

  • Determine if interruptions are coming into your sprints after the start.

Goal:

  • Did we complete all of the issues committed to at the beginning of the sprint and meet our goal?

  • Threshold goals can vary, but a standard goal is between 80-90% committed work completed during the sprint.

Starting Metric:

  • Committed vs Completed

Settings to Apply:

  • Using the “Descendants of” selector, choose the individual sprints that your team wants to track.

Efficiency:

Cycle Time

 

Why:

  • Understand if your team is consistently sizing and estimating work.

Goal:

  • Consistent upper-bound maximumns by issue type and story points.

    • For example, the smallest body of work the team creates, such as a sub-task, should have a median cycle time of under three days at most.

Starting Metric:

  • Cycle Time

Settings to Apply:

  • Series grouping to consider:

    • Story Points: Understand if the team is consistently pointing work of relative technical complexity and time.

    • Issue Type: Confirm the team has consistent definitions of work by issue type.

      • I.e. The smallest issue type (sub-tasks or tasks) should be work that is always completable within 2-3 days.

  • Aggregate by median to remove statistical outliers.

  • Filter out “Won’t Do” type states

Lead Time

 

Why:

  • Understand how quickly your team responds to high priority bugs, defects, and customer inquiries.

Goal:

  • Varies based on team social contracts and SLAs, but an example goal would be all P2 bugs should be resolved, on median, within five calendar days.

Starting Metric:

  • Issue Lead Time

Settings to Apply:

  • Filter out “Won’t Do” type issue/work item states.

  • Filter down to bugs, defects and/or customer requests.

  • Group by month or quarter.

  • Group series by priority/severity.

  • Aggregate by median to remove statistical outliers.

Issue Time in State

 

Why:

  • Analyze issue/work item process for bottlenecks

    • Finding median or average time in progress, wait states, etc.

      • For example, in progress states should often be under 24 hours. Make sure items aren’t bottlenecked waiting for QA resources, etc.

Goal:

  • Set an upper-bound per state. For example, our team should be sizing individual cards so that they don’t need to be in the active development state for more than two days, and our QA team has committed to picking up ready for QA cards within 36 hours.

Starting Metric:

  • Issue Time in State: Stacked Column Chart

Settings to Apply:

  • Aggregate by median

  • Group monthly

Pull request response time

 

Why:

  • Older pull requests cause merge conflicts, and are more difficult to review after they are no longer top of mind.

Goal:

  • Between one-half and three days.

Starting Metric:

  • Pull Request Response Time

Settings to Apply:

  • Aggregate based on average instead of median.

Coding Days

 

Why:

  • Understand how frequently your team is able to commit code. Use as a proxy for development coding time availability.

Goal:

  • 3-4 average days

    • Typical product/feature based dev teams.

    • Can be less for teams with architects, data scientists, QA, devops

Starting Metric:

  • Coding Days

Settings to Apply:

  • None Needed

Quality:

Currently Open Defects by Priority

 

Why:

  • Understand how quality is impacted by your focus on efficiency and speed.

Goal:

  • Varies by team. Allstacks recommends setting upper limits on number of bugs by priority open per team at any given time.

Starting Metric:

  • Issue Creation

Settings to Apply:

  • Measure time from “First Activity to Present”

  • Filter out all resolved or done states.

  • Filter to just bugs/defects.

  • Filter to your team’s relevant backlog/ownership.

  • Apply “Include Disabled Team Members” flag

  • Group series by priority or severity.

Bug/Defect Creation Over Time

 

Why:

  • Understand if your teams and customers have been finding more defects, and what severity they are encountering over time.

Goal:

  • Fewer issues created of higher severity/priority.

Starting Metric:

  • Issue Creation: By Type

Settings to Apply:

  • Series: By priority/severity

  • Include Disabled Team Members filter

  • Filter to the relevant backlog of the team via Advanced Filters

Readiness:

Refined Backlog Ready to Work

 

Why:

  • Determine the depth of refined work ready for the engineering team.

Goal:

  • Two to three sprints work of fully refined work.

Starting Metric:

  • Burndown: Remaining Issues by State (In Issue Count or Story point modes)

Settings to Apply:

  • Advanced filters to match returned cards to your team’s backlog.