Which Performance Metric is best?

Improve delivery performance by measuring the right thing

Performance metrics are the only reliable, objective way of proving that teams are improving their performance. We want to see the gains of our continuous improvement efforts, and here we can announce them to the world. But which performance metrics serve us best? Most teams measure Velocity and they stop looking for other metrics which might be more beneficial. Velocity as a metric has its challenges and detractors, and some don't believe that increasing Velocity should be the focus of the team, and I agree.

Measuring team performance can be either beneficial or potentially damaging, depending on what you choose to measure. Teams are heavily affected by how they are being measured, and their behaviour will change, so ensure that it’s a positive change.

This video goes into some detail about the various performance metrics available and what they mean.

Transcript:

Performance metrics can change the behaviour of software teams and therefore it’s really important how we select them, and what we use.  I want to talk about some alternative options to what you might be using and considering at the moment with the aim of allowing teams to work in a more iterative and agile manner while still demonstrating performance improvements.

We need metrics that will shine a light on where there are deficiencies in our processes, or ways of working, so that they can be remediated. We want metrics to be valuable and actionable.

So - what should we use?

Cycle Time and Lead Time are closely related

There are different schools of thought on how these should be measured.

I like to use Fin Goulding's definition, which we see here.

These two metrics alone are very powerful in telling you where there are deficiencies in the process. By making them the focus of improvement - which means to decrease them - teams will identify areas they can remove wasteful activities. The end result is that the customer, or end user, gets served faster, and are happier as a result.

I'm going to give you a metaphor to make it easy to relate to. If you're anything like me, then you don't mind paying 5 bucks for a good cup of coffee at a cafe.

When I go to my favourite coffee shop, they take my order, and it usually goes on a piece of paper and sits in a queue, waiting for the barista to finish what they’re currently doing. Lead Time starts from the moment that my order has been taken.  But the Cycle Time hasn't started yet. It doesn't start until the barista looks at my order, takes a clean cup and begins making the coffee.

The barista finishes the order by carefully placing a lid on my cup, and calls out my name. At that point both the Cycle Time and the Lead Time ends.

So the Lead Time is longer than the Cycle Time, and it tells us how long a customer must wait for their request to be fulfilled. Cycle Time tells us how long did it required to be made.

Throughput

Throughput is simply a count of how many work items were completed in a specific time frame. A cafe might measure the number of completed coffees in 1 hour, or 1 day. They might ask how they can increase Throughput during busy periods to keep up with demand during those periods and avoid losing customers.

Flow Efficiency is a ratio that tells us what percentage of time was a work item actively being worked on, compared to it waiting to be worked on. This ratio can tell us that there is low efficiency in our processes where items are waiting in a queue, or blocked for some reason. We measure the time spent in a wait state, and time spent in an active state, and compare the two as a percentage.

We can apply this to our portfolio pipeline and use that information to identify where work gets stuck in invisible queues in the organisation, and aim to lean out our portfolio process.

Lead Time Distribution measures the frequency with which work items are completed in a timeframe. In Agile, we often measure this in days. Looking at the graph, we might see work items being completed anywhere between 1 and 30 days. We can see that more items are completed on the lower end of the scale, and only a few on the upper.

If there is a large distribution, then the work is very lumpy, without a lot of consistency. This could tell us that some of the work is complex or challenging, or it could tell us that work items are too big for the team to complete. We can use this metric to identify if teams need help overcoming problems with the work items, or help breaking it down further.

We say that a team is more predictable when their distribution is bunched close together, and ideally it is at the lower end of the scale, meaning the majority of work items are easily completed, and consistently small.

Predictability is a good measure for telling us if teams are in a flow state. When teams are in a flow state, their predictability increases greatly and their throughput dramatically increases as a result. This measure is closely related to Lead Time Distribution. I will discuss it in a separate webinar, because it requires some explanation to make sense. Changing how you define "Predictable" away from meeting Project deadlines and constraints is one of the simplest shifts you can make towards greater agility.

Rate of Innovation is a term given to us by Elon Musk, and he says it is the only metric that matters, and is what sets companies apart.  Capabilities are a company’s competitive advantage, and not their assets.

Let’s look at the Rate of Innovation at Tesla. We know the Rate of Innovation at Tesla thanks to Joe Justice, who worked there and helped accelerate development.
"Tesla introduces 60 or more new parts into production and sale every day, 365 days per year. This includes the manufacturing processes, certifications, the embedded software, the support and maintenance procedures, the logistics, the supply chain management, and more. Tesla deletes 61 or more parts every day, the total part count continues to be simplified and complexity reduced." - Joe Justice

Tesla is now 8 times more profitable per car than Toyota, despite Toyota inventing their lean “just in time” production system last century.