Measures, Metrics, KPIs and OKRs

The Importance of Outcomes

In an agile organisation, work is organised - and evaluated - around business outcomes aligned with the overall strategy. Teams commit to improving outcomes but remain flexible about how to attain them. This is the essence of the agile mindset. It means not being bound to the execution of defined projects, but rather engaging in a continuous process of developing hypotheses about what customers value and validating them through disciplined experimentation.

In order to serve as a compass for agile action, outcomes must be measurable. Some outcomes are quite tangible, such as sales, profits, or customer retention. Others are less tangible, but equally important. Intangible outcomes may include more subjective measures of customer sentiment or employee attitudes. These intangibles drive behavior that in turn impacts more tangible, bottom line measures like sales or employee retention.

Measurement Frameworks

There are many approaches to measurement. In many cases, the differences are largely semantic. The most effective approaches start with a qualitative goal or objective that is aligned with a larger strategy. The qualitative objective is paired with one or more quantitative targets based on agreed-upon measures. This approach was first described as Management by Objectives by the late management guru Peter Drucker. In MBO, managers were educated about their superiors' priorities, and developed measurable objectives that aligned with them.

This approach was focused on the performance of the individual manager. Kaplan and Norton, originators of the Balanced Scorecard, adapted this to a more team-focused approach in the early 1990's. While balanced scorecard was adopted widely, it suffers from a "waterfall" mentality. Enterprise level objectives are set for a 3-5 year period and then cascaded down to successive lower levels in the organization. These objectives are organized into four perspectives:Learning and Growth (People); Process; Customer; and Finance. While this approach is quite thorough, it often results in an overwhelming number of measures to keep track of.

OKRs

Silicon Valley pioneers, beginning with Andy Grove at Intel, adopted a more agile approach to setting measurable objectives. This approach is called Objectives and Key Results, or OKRs. Objectives and measurable targets (KRs) have the same structure as in MBO, but are set, measured, and modified at a much faster pace. Objectives are qualitative statements of strategic intent, such as "Create an awesome customer experience!" and key results are quantitative measures with targets, such as "Achieve Net Promoter Score of 8.5 by end of Q3."

Typically annual OKRs are set at the enterprise level, with teams setting OKRs quarterly. Venture capitalist John Doerr learned this technique from Andy Grove while working at Intel and spread it to a number of other companies he invested in as a Managing Director at Kleiner Perkins, starting with Google.^{[For a complete history, see Measure What Matters by John Doerr]} It's important to note that team-level OKRs should be set by the teams themselves, not cascaded down upon their heads from above. This creates a much greater sense of engagement and commitment.

OKRs may align with higher level goals, or reflect local aspirations, such as removing a bottleneck with another team. At Google, for example, teams typically set 60% of their OKRs as "bottom up" goals, with the other 40% aligning top down.Once the OKRs are agreed to, the work can be accomplished in a typically agile manner, decomposed into weekly or bi-weekly sprints. Feedback conversations and progress reviews are frequent.

Google's approach is notable in the distinction they make between Aspirational (or Stretch) OKRs and Committed OKRs. It is not expected that a team will fully achieve an aspirational goal.

By its nature, it is innovative and the way to execute it is not clear in the beginning. 50% achievement is considered positive as long as something is learned in the process. There is no penalty for failure. These are called Moonshots. Committed goals, aka Roofshots, are things that must be done, in which a clear set of actions can be defined in advance. An example might be complying with a new regulation. It's also important that everyone's OKRs be transparent. This encourages spontaneous communications between teams that often result in major process innovations.

SMART Goals and KPIs

SMART is an acronym that has a number of explanations, such that there is no longer much agreement on what the letters stand for. My personal favorite is Specific, Measurable, Accountable, Relevant (i.e. to the strategy), and Time Bound. Some people say Achievable rather than Accountable, but that only applies to what Google would call a Committed OKR, not an aspirational one.

Don Sull of MIT has come up with a more up to date acronym, FAST. This stands for Frequently Discussed, Ambitious, Specific, and Transparent, and is very compatible with the OKR approach. KPIs tend to be quantitative measures that don't change much over time. For example, a hospital might always track measures of patient safety. Because these are often "hygiene" measures, they don't necessarily need to be tied to a larger strategic objective. Only if performance falls down would an objective be set to rectify it.