Measuring Productivity Using Stories

About a month ago, I attended some training on leading high performance teams. There I learned that a single well-defined metric that is perfectly aligned with the team’s performance can help to ignite their performance.  Among other things, this reignited my interest in actually measuring the productivity of my agile teams.

Despite many claims that productivity metrics are a fools errand (McAllister, Fowler, Hodges), I’ve been trying to measure it for years, at least ever since I came to Central 1, and possibly before. Without measuring productivity, the many easily grasped quality metrics are unbalanced, and the team can find themselves in constant-improvement mode, without actually producing anything new.  Without measuring productivity, how do we know that we are being strangled by technical debt?

For several years, I used the number of changes per hour of development. This got better when we jumped into agile with two feet back in 2014; prior to that, there was too much variability in the size of a JIRA ticket – they might represent a small bug fix or a whole project.  By the end of 2014, we were looking solely at the number of stories developed per hour (the reciprocal, hours per story, is more intuitive, but early on we sometimes spent time without producing any stories).

I was often asked why stories instead of story points.  The reason was value.  A very complex story would have a high number of story points, but might have little business value.  Stories, on the other hand, should be the smallest unit of work that can still deliver business value – a quantum of business value.

This metric was pretty good.  It had the immediate benefit of being cheap to produce – simply query JIRA and divide by time.  Moreover, the chart showed a beautiful increase of “productivity” as the team got used to working with agile.



But then a funny thing happened.  We were working on the new mobile branch ATM locator, and the project was producing stories just fine, but it was never concluding.  The problem was in the nature of the stories.  Instead of meaty stories like,

As a user I would like to search for the closest ATM so that I can go get money.

many of them were more like

As a user, I would like the search box to be titled “search” so that I know where to put my query.

Clearly, not all stories are created equal.  I don’t think the team was deliberately gaming the system (there was no benefit if they did) and small stories are a hallmark of a healthy agile team, but surely we cannot ascribe the same level of productivity to a team that is changing the value of a label as one that is enabling search. More to the point, the team was not completing the project!

I feel that the unevenness in story value probably averages out over a sufficiently large team.  However, measuring over the larger team has little benefit in terms of motivating a single agile team.  Across the larger development organization (Central 1 has about 50 developers and 20 testers), we might expect to see an effect if we make a change for everyone, as we did when we moved to adopt agile in early 2014.  However, because the values are not steady, it takes four to six months to be sure of a trend.  On the other hand, it is very difficult to dissect what is happening if no change has occurred, but a trend is detected anyway.

Looking forward, there is a promising-looking blog post from Isaac Montgomery at Rally.  It has the benefit of measuring true productivity, but requires valuation for initiatives, which at Central 1 would be difficult.



Tags: ,

One Response to “Measuring Productivity Using Stories”

  1. How Productive was our Sprint? A Proposal | ReneGourley@work Says:

    […] Working in a Digital World « Measuring Productivity Using Stories […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: