Measuring The Transformation

Measurable Business Outcomes

Agility is not a set of methods or practices or frameworks. Adopting Scrum or SAFe does not guarantee you will deliver products to market any faster. Many organizations are “doing agile” but not realizing the sought after business outcomes.

Agile transformations are expected to be more than just incremental improvements to performance. A transformation must take an organization to a fundamentally different place, and deliver a significant leap forward in business performance – speed of delivery, customer excitement, employee engagement, revenue growth and margins. (Otherwise, why take the risk)?

Agility is a means to an end, an enabler of organizational and business performance. Agile Transformation, as part of an organization’s overall business strategy, is pursued in order to achieve business outcomes like:

• Improved operational performance: The ability to deliver products to customers more rapidly and reliably. Being able to quickly respond to value-creating opportunities and threats in the market.
• Higher customer satisfaction: The ability to attract and retain customers by providing higher value products and services with superior user experiences.
• Improved employee engagement: By providing employees with more autonomy, and opportunities for learning and advancement.

These objectives contribute to the ultimate business goal of increased revenues, reduced operating costs and improved margins. Organizations cite goals like these as motivations for undergoing agile transformations. However, many are flying blind throughout their journey. They do not know what progress looks like or how to measure it. Or, collect measurements that are completely disconnected from target business outcomes.

Agile Transformation should result in measurable improvements to things like:

• Product Delivery Performance
• Customer Satisfaction
• Employee Engagement

The challenge is connecting initiatives of the agile transformation to these outcomes.

New Capabilities Needed

Fundamental changes in organizational capabilities are likely required to achieve the goals defined for the transformation.  A business capability is what a business does to generate value. Capabilities should define what the organization needs to do – not how. The how part is expressed in terms of practices, tools, applications – these things change over time. Selected capabilities need to have a measurable impact. If you can’t measure the impact of making a change, there is no way to know we have improved anything. We also need to distinguish between capabilities and practices. The distinction may appear nuanced however capabilities describe what is needed, whereas practices describe how. Capabilities lead to measurable outcomes (like Lead Time reduction), whereas the impact of individual practices like Pair Programming, for example, can be difficult to correlate with measurable improvement. A capability mapping exercise is a good way to show how business goals, required capabilities, and supporting practices and applications/tools are connected. This exercise can help us align process and tooling changes to capabilities. Capability Mapping shows how business capabilities plus supporting practices define the activities, systems and processes that must occur in order for a business to meet its goals.

Defining the capabilities needed to support the transformation and the enabling processes should be part of the transformation plan. The why, what and how of the transformation plan must demonstrate consistency.

What to Measure

Organizations must be able to connect the impact of specific transformation actions to their target outcomes. A framework is needed that connects an organization’s vision and strategy with clear and measurable objectives. OKRs (Objectives and Key Results) are a good way to define objectives for the transformation and to measure progress towards their achievement. Agile transformations will have many uncertainties and hence will require continuous inspection (measurement) and adaptation. It is crucial to maintain a feedback loop, be open to making adjustments, and to continuously learn and improve based on the observations.

Progress towards objectives can be tracked via a set of basic OKRs. For example:

• Objective: Accelerate Time-to-Market:
  • Key Result: Deployment Frequency increased by 50%.
  • Key Result: Team delivery predictability > 80%
• Objective: Provide Customers with Highest Product Quality:
  • Key Result: Customer-reported defects reduced by 33%.
• Objective: Improve Resolution Time on Post-Release Defects:
  • Key Result: Customer-reported defect resolution time reduced by 25%
• Objective: Best in Industry Customer Experience:
  • Key Result: Net Promoter Score (NPS) > 8.0.
  • Key Result: Customer Retention Rate (CRR) > 80%.
• Objective: Best Place to Work for Employees:
  • Key Result: eNPS Score > 8.0.

Bottom-Line Impact

Agile transformation must have an impact on 3 fundamental dimensions of any enterprise:

• Product: The organization’s effectiveness in delivering its products or services.
• Customers: The organization’s ability to attract customers and retain them.
• Employees: The organization’s ability to attract employees and keep them motivated.

Improved performance in each of these areas contributes to ultimate business outcomes of increased revenues, reduced operating costs and improved margins.

Setting Up a Measurement Framework

Effective transformation governance must include data-driven inspection and adaptation. Basic steps in setting up a measurement framework likely include the following:

• Define overall outcomes that are being pursued by the transformation.
• Define major objectives to achieve each outcome.
• Determine a way to measure progress on achieving objectives – OKRs and targets.
• Build the necessary instrumentation for data collection and reporting.
• Set up a system of reviews to evaluate progress towards the objectives.

Make Measurements Visible

Get the objectives and associated metrics onto a corporate website and make them visible to everyone. Make it easy to see what improvements everyone is working on. Review at every opportunity – Town Hall meetings and the like. Objectives are reviewed on a regular basis, and periodically reset, for example every quarter. Like everything else in agile, progress is made empirically based on the principles of transparency, inspection and adaptation.

Specific initiatives and actions for achieving each objective should be defined. Treat these as experiments and be prepared to try new things if sufficient progress is not made. The important thing is to relate initiatives associated with agile transformation to business outcomes and to demonstrate the benefits over time.

Automate Measurements

Every metric used to drive the transformation should be well-defined and visible. Metrics should measure outcomes vs. activities, and their role in transformation decision-making should be understood. Generation of these metrics should be automated. Automation brings consistency, and ensures all metrics are up to date.