+49 (0) 89 58803-1000

About Metrics, Their Imbalance and a Magic Word in Agile (Product) Development

“When something is measured, it is being completed!”

I am quite sure, you do not read this sentence for the first time. Do you agree? Well, I am torn. If we assume that this statement is true, is it then also true within agile (product) development? As often misunderstood by the public, agility does not primarily mean “speed” but rather flexibility. Flexibility, in turn, describes in this context the ability to react to changing requirements. In times of agility the following questions are being asked: Are metrics still required? Do we still have to be flexible when it comes to key figures? Do we have to adapt to changing requirements?

The necessity of metrics and the role of trust

The question whether we need metrics in an agile environment can be answered quite easily. To put it partly in Ken Schwaber’s words: “If the product is the best possible software in consideration of cost, functionalities, time and quality”, then you can assume that metrics are under no circumstances irrelevant but immanent. To be able to consider the “magical” factors, you have to measure them initially.

Before I start to go into detail about the questions raised, you will please forgive me that I will now briefly “kidnap” you. It is no secret that trust plays a central role in an agile environment. Without beating about the bush: Agile teams do not work without trust! Maybe you ask yourself now, what this finding, which is admittedly not really new, has in common with metrics? A counter question: What happens with trust in a Scrum team, when the Scrum Master starts to collect all metrics available and, in case of doubt, produces a report about these metrics? The “health” of a team is based on trustful relationships. When considering the question concerning the relevant metrics, trust plays a role insofar as all persons involved should be aware of the fact (in the sense of maximum transparency) for which purpose exactly a key figure is calculated.

Ok, which kind of metrics become especially relevant for an agile approach? I hope you do agree with me when I am writing for example that the numbers of lines of code definitely cannot measure the productivity of a development team; and that productivity can be evaluated most suitably based on the functionalities which are hopefully being developed. Furthermore, I assume we all agree, when I also write: From the company’s perspective, a simple look at the recently developed functionalities cannot be enough. We do not collect metrics for their own sake. They rather help to achieve certain targets. Generally speaking, most companies want to uncover the work progress, measure the quality of work products and, therefore, be able to make good decisions in the end.

Velocity, Velocities and the Velocity Factor

Presumably, the most widely used metric is “Velocity” – whether the Sprintvelocity on team level or – in the event of scaled Scrum – the program, product or portfolio Velocity. The velocity of newly delivered functionalities is measured and this seems to be relevant on all levels of a modern company. But relevant for what? Unfortunately, the accusation that the velocity is considered only to monitor the teams is not entirely wrong. However, from my point of view this accusation is quite unrealistic. Without a doubt, key figures for velocity can be misused for punishing low velocity and for rewarding high velocity. However, I write the word “misuse” on purpose, since actually one of the advantages of a continuously measured velocity in an agile context is to be able to make valid predictions. In contrast to the classic project plan, such a prediction does not necessarily have to be absolutely accurate, but is, nevertheless, essential as a part of the communication between Business and IT. (If you like to read more about simulated accuracy of classic planning and about the agile estimation, please click here!).

In addition to the usual Velocity described above, further Velocities can also be measured. As you know, Scrum focusses on added values for customers. Similar to complexity, this added value can be evaluated in points per User Story. They are called “Value Points”. By means of “value points”, the Product Owner evaluates the added value that can be generated by the User Story. Ok, so, why are we not measuring the Value Velocity, too? This key figure helps the team to make the added value – accumulated and realized per iteration – visible for each customer.

Additionally, two easy and frequently collected key figures are the numbers of Story Points, which are actually delivered in the end of a Sprint and the number of points that was defined and agreed upon in the beginning of the Sprint phase. By setting both figures in a relation with each other, a factor is created (i.e. Story Points done / Story Points committed).Let us call this factor “Velocity Factor”. It provides insight, which shows what percentage of the Sprint target has already been put into effect by the team. The closer the Velocity Factor gets to one, the better the team estimates its own performance. A good Velocity Factor is a prerequisite to provide stakeholders with a realistic estimation about the functionalities deliverable in a certain time and to estimate how many Sprints a team presumably needs in order to deliver a certain functionality.

Test – We Automate and Cover

Test automation is another popular metric, yes even a universal remedy, if you want to believe some of the literature. It helps to clear stumbling blocks and time-eaters (e.g. troubleshooting) by keeping the code of automated tests modular and clean and by avoiding regression. Of course, it only works in case of a high test coverage. A 90 % test coverage does not automatically mean that 90 % of the code are “clean”. However, a high test coverage must mean anything good, doesn’t it? Is it really logic that an especially good code shows an especially high test coverage and does a high test coverage – in turn – automatically prove a well-written code? Basically, a broken code would also be possible in case of 100 % test coverage. To be honest, I do not have the necessary IT expertise to answer this question from a technical point of view. Nevertheless, I raise the question at this point since I have not found a satisfying answer yet. Should you know it, please let me know!

Imbalanced metrics, quality and happy teams

In addition to the metrics described as an example, there are even many more. Since I do not claim to know all metrics in detail and I do not want to bore you with a long list, let me just ask one last question now: Do you agree that a collection of metrics in itself improves the measured performance? I do not, especially, if the collected key figures are purely a representation of performance. It is a possible effect that performance is superficially increased but the question why performance was increased is ignored. It is possible that the introduction of a metric (e.g. Velocity) has a motivating effect and makes it possible for an agile team to becomes more efficient. It might, however, also be possible that the metric builds up pressure and the awareness of a team being monitored leads to the situation that the team delivers more quickly but at the expense of quality! In order to reduce the risk of such an effect it might help to measure the following metric for instance: “the well-being of the team”. Slightly esoteric, but no less important than other metrics. Well-being is measured on a scale or with a simple yes-no query. I would even go so far as to claim that well-being (already known as satisfaction or luck) determines all other metrics. The happier the team, the higher its productivity. An essential difference of well-being compared to the other metrics mentioned above is that well-being in itself lacks objectivity and that it is – strictly speaking – not a metric. However, this finding does not change the usefulness of the result for each team, since also things that cannot be measured in an objective way can be improved.

Leave a Reply