Software Delivery Metrics

This is a post from 2014 stuck in my drafts. Be free little post… be free.

We have been pondering metrics for software delivery at work. Let me tell you, trying to hammer down a core set of global metrics for an organization with thousands of developers is not an easy task. Fortunately, in my personal projects I am only concerned with:

  • How many defects are reported in production.
  • How fast are we fixing production defects.
  • How many production defects are recurring or repeat offenders.

Can there be more metrics? Absolutely, but until I have a good handle on these I don’t want to complicate things by tracking anything that doesn’t have a direct affect my customers. Having 5, 10, 20…or more metrics that I actively track would make me over analyze and spread my focus to wide. Keeping it simple and focused on the metrics that bring the most insight into keeping my customers happy with my product is most important.

Would this limited set of metrics work for every project, every company…no. My metrics are optimized to the goals of my small product and company. You have to find that thing that is most important to your company. This is where it get’s difficult. There are so many opinions of what a good metric is and people want to advocate the metrics that have worked for them. The answer to large scale metrics projects may be a focus on achieving a core set of goals and only having metrics that have a direct correlation with the goal while having relevance in every part of the company. Easier than it sounds, but I believe this would force the scope of the metrics program downward. Less metrics is a good, good thing.

In fact, I believe that a burgeoning metrics program should focus on one thing at a time as you ramp up. Choose one problem to fix in your software delivery and find a metric that can shed light on a possible way to fix it. If you have a problem with delivery time, maybe some type of process flow type metric would benefit you if you follow Kanban. What you want to do is optimize your metrics for your particular problem space and there isn’t a secret formula or magic bullet that someone can write in a blog to get you there. You have to try something. Pick a relevant metric throw it at the wall, if it sticks, run with it and find another one.

Once you have a metric, get your benchmark by querying your current data to see where you are with the metric. As you know, the benchmark is your measuring stick and the point where you measure your good and bad trends from. Once you have your benchmark, then develop a tracking system: who to collect, store and report on the metric. Begin tracking it and implementing programs to improve the metric. Follow the trend of the metric to see how your changes are affecting the metric. Then when you have a handle on how the metric works for you then you will have a framework to develop additional metrics. You can call it the Minimum Viable Metric, if you will.

The point is, if you spin your wheels analyzing what metrics to use, months will roll by and you will be no better. Precious data would just be passing right by you. Start today and you may find yourself with a wealth of actionable data at your disposal and the means to roll out more metrics.

Leave a comment