Delivery At Scale 5: Improvement

The Improvement System

Summary

All improvement should be connected to business outcomes.
Improvement frameworks can be leveraged to ensure alignment between agile initiatives and business objectives.
Improvement initiatives can be managed in the same way as the delivery of new product features: Add to Program Backlogs and apply program governance.
A small number of KPIs and process metrics is sufficient to evaluate the performance of core delivery processes, and to point at where improvement is needed.

Agility As A Means To An End

Agility is a means to an end, an enabler of organizational and business performance. Businesses ultimately exist to deliver a return to their owners or shareholders, and successful organizations attempt this in a highly systematic way by having a clear goals and strategies to realize them. Operational Agility, as part of an organization’s overall improvement strategy, is about having the ability to perform core processes, like software development, with superior speed, productivity and flexibility. Operational agility is an enabler of superior business outcomes.

Many organizations use frameworks like Hoshin Kanri, Balanced Scorecards, or Value Driver Trees to support the pursuit of strategic objectives. These frameworks serve to integrate an organization’s vision and strategy with clear and measurable performance objectives. KPIs (Key Performance Indicators) are used to define and measure progress towards these objectives. Reviews and feedback loops examine progress of improvement initiatives and provide opportunities to continuously adjust and optimize.

Improvement As A System

Improvements at all levels in an organization must be set up and operated as a system using a process-driven approach. Process-driven means that the system will have defined inputs, outputs, and feedback mechanisms, and will run on a cadence that reflects a basic PDCA cycle framework.

Plan: Establish a set of objectives that contribute to realization of the vision. Identify specific initiatives or actions that will have a measurable contribution towards meeting these objectives.
Do: Take action and implement the initiatives.
Check: Measure progress towards objectives, and evaluate the impact.
Adjust: Adjust plans and make course corrections to better converge on objectives.
Repeat on a cadence that is appropriate for the business environment.

Business Transformation Frameworks

Operational agility may be one element of an overall business transformation strategy. Operational agility seeks to improve internal process capabilities, including software delivery performance.

Those familiar with the Balanced Scorecard framework (Ref. 2) will be aware of how objectives are collected into 4 areas of focus, or the so-called 4 perspectives, namely:

Financial Perspective: How do we look to our shareholders?
Customer Perspective: How do customers see us?
Internal Business Processes: What internal business processes must we excel at?
Learning and Innovation: How will we sustain our ability to continuously improve and grow? How do we keep evolving our skillset and foster a culture of innovation?

At the overall business level, this process may look something like this:

Strategy Formulation With Continuous Adaptation

If achieving high levels of operational agility in software delivery is a strategic imperative for an organization, then the process to achieve that objective needs to be defined and managed. Setting up such a system likely includes steps similar to the following:

Develop a roadmap and strategy for agile transformation
Construct a Value Driver Tree connecting strategic objectives with proposed initiatives that realize it.
Determine KPIs and targets.
Build the necessary instrumentation for data collection and reporting.
Set up a system of reviews to evaluate progress towards the objectives.

Improving performance in one area can support or enable better performance in another. Improvements are mutually reinforcing.

Operational Agility

Operational Agility is about improving an organization’s core process capabilities to enable superior business outcomes. Strategic objectives are pursued by means of various initiatives, and progress is measured via KPIs. KPIs are outcome metrics (lagging indicators). Value Driver Trees are a useful way to visualize the chains of cause-and-effect that connect operational improvements to business outcomes. Operational agility can be measured by delivery process metrics (leading indicators).

Here is an example (partial) Value Driver Tree:

Operational agility is not solely based on delivery processes. Delivery teams which are cross-functional and highly autonomous are empowered to make faster decisions. Thus team structure is also a significant determinant of operational agility.

The Improvement System should be providing the organization with data on how well it is performing across a number of dimensions. Data systems should support continuous improvement efforts and provide the organization with clarity on basic performance metrics.

Make Improvement Initiatives Visible

Initiatives, whether aimed at developing new product capabilities, or at improving the delivery performance of the organization, should be captured in product and program backlogs. This brings visibility, helps ensure organizational support and alignment, and enables us to measure progress.

Thus, improvements are built into programs just like any other deliverable. They are owned by the delivery teams. Improvement effort can be quantified, and progress can be evaluated at regular program reviews – not at a separate event. This makes all improvement-related activities part of the standard operating framework of the organization, and ensures that improvements are seen as critical business deliverables, and not nice-to-have things that happen in our spare time.

Operating The Improvement System

Whether at the strategic or operational levels, any basic improvement framework must include a regular review of progress vs. objectives. This is the core of the feedback loop – the process of checking and adjusting, or inspecting and adapting.

Program Reviews will now be highly data-driven, and include a mix of program metrics, and process metrics. What we are looking for is metrics that support the goal of goal of continuously improving an organization’s software delivery performance.

Program Progress Metrics:

Feature completion: Percent Done by feature (in stories or story points).
Program Completion: Burn-Up in done stories or story points for overall program
Program Impediments: #Open, Closed, etc.

Process Performance Metrics:

Feature Intake Cycle Time: The average time from feature request until it is considered ready for development
Feature Development Cycle Time: The average time to completed development of a feature to a definition of done.
Feature Lead Time: Average time from feature request to feature release.

These metrics can be improved by identifying sources of waste and delay in the delivery process. This leads to improved end-to-end delivery lead time, which has a direct impact on revenues and margins.

Further, the relative priority of improvement initiatives should be directly proportional to their impact on reducing lead time.

The specific data brought to a program progress review will be specific to that business and it’s current plans, but it will include a set of KPIs linked to objectives, with data showing progress for each metric:

In practice, the dashboard or scorecard will be a mix of KPIs and process metrics. What we are interested in is how well our 2 major processes – Intake and Development – are performing, and how those processes are enabling better outcomes for customers and the overall business. At minimum, metrics and KPIs we will want to track should include:

The objectives of any agile transformation at minimum (and associated metrics/KPIs) include:

Higher Productivity (Velocity)
Faster Delivery (Lead and Cycle Times)
Higher Employee Engagement (Employee Survey Scores)
Better Customer Satisfaction (NPS Scores)

Product or Program Backlogs constructed using our Intake System will contain a mix of product features and improvement initiatives necessary to achieve product and process improvement goals.

And just as we can construct value driver trees to demonstrate how work in the guise of new feature development is directly connected to business outcomes, so we should be able to show how any improvement activities contribute to the goal of reducing lead time. Further, the relative priority of improvement initiatives should be directly proportional to their impact on reducing lead time.

Lead Time vs. Cycle Time

Lead Time and its related measure, Cycle Time, are important concepts in lean and agile, and both can provide insights into how a system is performing and how to improve it. In terms of our governance model, these metrics can be illustrated as follows:

Lead Time refers to the amount of time it takes for work to move from initial business request to release of working software. It encompasses not only all of the product development activities of planning, implementing and testing, but also all of the business-related tasks of definition, analysis, estimation and prioritization.

Cycle Time measures the efficiency of a system in processing units of work. For our Product Development System it measures the effectiveness of transforming product backlog items into working software.

Lead Time encompasses both the time needed to for work to be processed (development cycle time), and also the time needed for work to be made ready for the development system (work definition cycle time).

How do we improve lead time?

Improve cycle time for each part of the system. In the language of lean this involves the continuous identification and removal of waste in all its forms: defects, hand-offs, queues, unnecessary meetings or documents, technical debt, and other impediments. Agile development teams will use the retrospective process as a primary source of improvement. The same applies to the product definition part of the system, where the goal should be to get requested features into a state of readiness and onto the product backlog with minimum delays and without overloading the capacity of the development system.
Increase the capacity of the system. Add more delivery teams to service the product backlog. This needs to be done with care to minimize the additional overhead associated with more elaborate planning and synchronization of multiple teams.
Impose a WIP limit on the number of new feature requests accepted for analysis. Such a limit would be refined over time to converge on a stable intake cycle time. However this may result in sometimes telling stakeholders that you have no more capacity to deal with their requests. That may be OK in businesses where end-to-end predictability is important. However if stakeholders find themselves frequently waiting to get requests processed, that should lead to a discussion about adding capacity to address the problem.

Improving Intake Cycle Time

The agile literature is filled with writing about scrum, and sprint planning, and story points, and pair programming and all manner of methods, tools and techniques of agile development. Intake is frequently ad hoc and unpredictable. Making it more agile at least involves having a basic workflow, reducing WIP, and operating on cadence. Then actually measuring the performance of the intake process and identifying opportunities for improvement. One could propose an additional branch of the value driver tree connecting business outcomes of revenue increases and cost reductions to a strategic driver of lead time reduction and from there to actions involving application of WIP limits and other actions that improve intake flow and throughput.

References

Baskarada, Sasa (2020). “The Seven S’s of Organizational Agility”.
The Balanced Scorecard—Measures that Drive Performance. HBR Magazine, (January–February 1992). Robert Kaplan and David Norton.