Skip to main content

Team Metrics - Case Study

Let's look at an info-graphic of a beginning team's metrics and use this as a case study in Scrum Team Metrics.

Description of charts:

Burndown chart - a daily count of the number of task units (aspirin is this teams selected units for task estimation) not done.  This includes the task yet to be started, and task in process.

Tasks in Process - a daily count of the number of tasks in process.

Tasks Done - a daily count of the number of tasks that are done.

Stories Done - a daily count of the number of Stories that are done.

Velocity - the empirical measure of Stories that are considered done by the team and accepted as done by the Product Owner during the Sprint Review.

The Back Story on this team:

This team had been attempting to do some form of ad-hoc Scrum / Kanban with little guidance and understanding of the process.  The Kanban aspect came from the company's tooling (RTC) template - not from any real practices the team was implementing.   After some weeks of observations and workshops with the team - they decided to "hit the reset button" on doing Scrum.  Sprint One in the info-graphic is the first sprint right after a week long workshop on learning Scrum practices and principles.  Key to this team's adoption of Scrum is their adoption of a physical task board (see also Elements of an Effective Scrum Task Board).

Observations on Sprints:

Sprint One - Started with many stories from past sprint that were not yet done - as the team had no empirical data of velocity we guessed at how many stories we could complete in the 2 week sprint, and chose 15 stories.  At this point we had 4 product silos where people we working within the silo to deliver the stories - very little cross team collaboration.

Rules silo
Sprint Two - Tear down the silo walls - the team decided that the original silos of working was harming a long term desire of cross-functional team members - so a removal of the silo walls (tape on the scrum task board) happened.

Sprint Three - Enforced the use of empirical data to constrain the team's selection of how many stories to bring into the sprint (team select top 5 stores and finished all of them).

Sprint Four - Team planed for 30 points of stories but finished early and pulled in additional stories and finished them within the sprint.

Objectives for the Team:

This teams objectives for hitting the reboot button on a scrum implementation was to achieve a consistent level of reliability to deliver value (stories) to the business.  Also to maintain and supporting the existing 4 products line internal organizational clients, and transitioning tacit knowledge from several remote employees to the team and increasing cross-functional capabilities of the team members.

Commentary on Metric Charts:

Burndown - Sprint 1 and 2 task burndown charts show that the team started with around 100 aspirin and discovered between 50 and 100 aspirin more by doing the work - but didn't finish the 15 stories and left lots of stories started but incomplete at the end of the sprint.  In sprint 3 and 4 the team had developed the ability to forecast the proper amount of work to pull into sprint planning and were able to deliver the completed stories.

Tasks in Process - this simple metric showed that the team of about 8 people were consistently task switching.  There are many "reasons" (excuses) for this behavior, and it is a hard habit to correct in this era of high utilization rate driven management.  Just tracking this metric had little effect on the teams behavior - however we had empirical data that other practices (avatars, re-estimating in process tasks, etc.) had a positive effect upon this metric over several sprints.

Tasks Done - this metric is redundant for a team using a traditional sticky note task board.  In general this reflects the sprint burndown.  It does point out for this team that tasks done stalls out when there support tasks flare up, as these support (maintenance and production, M&P) issues require task switching to the more urgent unplanned work.  Reflecting upon this metric lead the team to start tracking the planned tasks separate from the urgent support tasks in our burndown chart for sprint 5.

Stories Done - an interesting trend shows up in this simple to trend metric.  The team was capable of finishing 5 stories, regardless of how many they planned.  In sprint 3 when the team constrained the planning to the empirical evidence (~28 points, 5 stories) they had there first successful sprint (on time, on budget, with planned scope).

Capabilities developed by the Team not shown in these Metrics:

Tasking - working toward tiny tasks.  Within the first two sprints the focus was to develop the ability to task stories.  Several synergic practices lead to this capability - re-estimating each time the task is touched in stand-up; recognizing that task that last for several days are way-too-large;  learning to decompose tasks that are too large; realizing that doing work leads to discovery of new tasks that need to be recorded and added to the board.  See Also: What belongs on the TASKS board?

Single Piece Flow - working on a task until it is done.  Smaller task effect this behavior in a virtuous manner.  Re-estimating each day makes the antithesis of this pattern apparent, and also offers the opportunity for team members to recognize when help is needed.  The use of avatars on the story tasks reinforces the practice of lowering work in process and reducing task switching.

In Sprint 5 the team decided to move from a 2 week time box to a 3 week sprint. The charts also show the support (M&P) tasks tracking independently of the planned tasks and the new chart at the bottom (M&P task vs Planned task deltas per day) indicates the inverse relationship of the priority shifts the team has to deal with.

Next Objectives:

Develop the capabilities to deliver agile release plans and forecast feature release time frames for business coordination with other teams that depend upon the infrastructure product lines developed by this team.

At the team coaching level an objective is to measure cycle time of stories within scrum teams.

See Also:
Metrics for a Scrum Team - 10 suggested metrics and examples
Measuring Process Improvements - Cycle Time by Mishkin Berteig, June 2008

Post a Comment

Most Popular on Agile Complexification Inverter

Exercise:: Definition of Ready & Done

Assuming you are on a Scrum/Agile software development team, then one of the first 'working agreements' you have created with your team is a 'Definition of Done' - right?

Oh - you don't have a definition of what aspects a user story that is done will exhibit. Well then, you need to create a list of attributes of a done story. One way to do this would be to Google 'definition of done' ... here let me do that for you: Then you could just use someone else's definition - there DONE!

But that would be cheating -- right? It is not the artifact - the list of done criteria, that is important for your team - it is the act of doing it for themselves, it is that shared understanding of having a debate over some of the gray areas that create a true working agreement. If some of the team believes that a story being done means that there can be no bugs found in the code - but some believe that there can be some minor issues - well, …

Elements of an Effective Scrum Task Board

What are the individual elements that make a Scrum task board effective for the team and the leadership of the team?  There are a few basic elements that are quite obvious when you have seen a few good Scrum boards... but there are some other elements that appear to elude even the most servant of leaders of Scrum teams.

In general I'm referring to a physical Scrum board.  Although software applications will replicated may of the elements of a good Scrum board there will be affordances that are not easily replicated.  And software applications offer features not easily implemented in the physical domain also.

Scrum Info Radiator Checklist (PDF) Basic Elements
Board Framework - columns and rows laid out in bold colors (blue tape works well)
Attributes:  space for the total number of stickies that will need to belong in each cell of the matrix;  lines that are not easy eroded, but are also easy to replace;  see Orientation.

Columns (or Rows) - labeled
    To Do
    Work In P…

Webinar: Collaboration at Scale: Defining Done, Ready, and NO.

I was invited to participate in a Scrum Alliance Webinar.  Maybe you would like to listen to us in a discussion of techniques to collaborate at scale (remotely and with many people).  The topic is one that I've got some experience in discussions - yet I never seem to get to done...
Collaboration at Scale: Defining Done and Ready and NO for Distributed Teams
With Joel Bancroft-Connors, Agile Organizational Coach; David A. Koontz, Agile Transition Guide; and Luke Hohmann, CEO and Founder of Conteneo, Inc.

14 February 2018 11 a.m. ET (USA).

The Scrum Guide is pretty clear on the criticality of the definition of Done: "When a Product Backlog item or an Increment is described as "Done," everyone must understand what "Done" means. However, the Scrum Guide ALSO says that the definition of Done can "vary significantly per Scrum Team." This leads us to examine when and how the definition of Done should vary, how distributed teams should cr…

A T-Shaped 21st Century Knowledge Worker

Knowledge workers in the 21st Century must have many areas of deep knowledge, while also be capable of collaboration across multiple other domains with dissimilar T-shaped individuals.  This description of a person is a metaphor.  Compare it to the shape of the "I" in the classic saying there is no "I" in Team.

I first read about Scott Ambler's term "Generalizing Specialist" - but it's so hard to remember the proper order of the words... get it backwards and it has an inverted meaning... T-Shaped is easier to remember. 
A generalizing specialist is someone who:
Has one or more technical specialties (e.g. Java programming, Project Management, Database Administration, ...). Has at least a general knowledge of software development. Has at least a general knowledge of the business domain in which they work. Actively seeks to gain new skills in both their existing specialties as well as in other areas, including both technical and domain areas.  General…

A FAILURE to Communicate

I was working with a failing team some time ago.  I use "failing" to describe the outcome of the team - not the people on the team.  Are you OK with that description?

An issue arrose in the stand up - a team member that was to verify the quality of a procedure did so and reported that there were a few records that didn't match expectation in the data set.  Upon inquire the number of records not matching was over 2000.  Most people acknowledged immediately the exaggeration - I could tell by the laughter.  After about 10 minutes of discussing the details of the problem - it appeared the team had a handle on the specific situation.

I stopped the discussion and inquired if they could name the impediment.  One team member did a great job of describing the impediment as a _communication gap_.  Wonderful - I could work with that - the problem had a name and it didn't include anyones Proper Name.

"If the problem has a first name; we are going to have a problem."