Agile Transformation – A Basic Framework

“Change is not mandatory. Neither is survival”.

W. Edwards Deming

Agile Transformation is a means to an end. Agility is an enabler of superior business outcomes. Organizations expect that Agile Transformation will enable accelerated delivery cycle times, reductions in operating costs, improvements in product quality, faster strategy pivots and more satisfied customers. However, many transformation initiatives become protracted, expensive, and ultimately stall with little of real value accomplished. Many large agile initiatives not only miss their goals but also cause organizational disruption and adversely impact product development programs. Reasons for failure include lack of organizational alignment on goals and strategy, insufficient skilled change agents to support the initiative, lack of accountability, and inadequate or inconsistent governance.

An Agile Transformation may be one component of a broader Digital Transformation (or technology modernization initiative) which includes things such as switching to a cloud infrastructure, AI-driven sales improvement, and automation of manual processes like customer service. No two transformations will be exactly the same, but their overall business objectives will be fairly common: accelerate financial growth,  gain market share, become a faster, more responsive organization delivering more value at less cost.

Whatever the specific goals of any transformation, the application of agile principles to the transformation process itself is much more likely to deliver successful outcomes. This means employing an incremental, adaptive approach to manage risk and maximize goal achievement.

Transformation is often an expedition into the unknown. At the outset, we don’t know what we don’t know. A transformation strategy built on the empirical pillars of inspection, adaptation and transparency makes a lot of sense. That is:

• The transformation approach is adaptive in contrast to following a fixed plan.
• Progress is made in small increments followed by inspection with adaptation for the purpose of learning, risk-reduction and course-correction.
• Inspection is enabled by transparency, that is, problem-solving and decision-making is based on facts and data.

This makes an agile approach to any type of transformation more flexible and reduces the risk and cost of failure.

The Core Idea

• Incremental change
• Evidence-driven
• Transformation Backlog, modeled as work items
• WIP-limited, Implemented when delivery capacity permits
• Measured via dashboard
• An independently verifiable Definition of Done
• Adaptive: Build–Measure–Learn mindset

This prevents:

• Big-bang framework adoption
• Organization-wide disruption
• Change overload
• Cultural theater

The Risk: Turning the Transformation Backlog into a Checklist

The goal is not: “Complete all transformation stories.”

• Each transformation backlog item must be considered: A hypothesis about improving system performance and have a measurable outcome.

For example:

• implement CI/CD pipeline to reduce feature cycle time by 30%.”

Now it is measurable. Now it is testable. Now it can be stopped if ineffective.

An Agile Framework for Change.

The PDCA (Plan-Do-Check-Adjust) cycle is an adaptive approach for change management and continuous improvement, and provides a sound framework for agile transformation. 

Plan:

• Define the vision and objectives for the transformation. A compelling vision helps inspire employees at all levels to commit to the transformation. Objectives should be stated in terms of measurable business outcomes. Leaders play a critical role in defining and communicating the vision for the transformation and helping to ensure that transformation objectives are strategically aligned with business priorities.
• Devise a strategy/plan. Define the overall goals for the transformation and from there devise a plan in the form of a Transformation Backlog. A Transformation Backlog is an ordered list of everything needed to realize the goals for the transformation. The contents of this backlog may evolve over time, but we want to establish a baseline to get started. Progress can then be measured in terms of how much of the transformation backlog has been implemented by each team.
• Setup a Governance Framework. Governance is the Roles/Events/Artifacts that will be used to steer the transformation. This includes defined roles/responsibilities, a cadence of events for planning, progress reviews, and the key data and artifacts that will be used for measuring progress and decision-making. Skilled change agents are needed to lead this process. 

Do:

• Execute. Execute incrementally. The Transformation Backlog can be implemented incrementally across the organization. Different teams may proceed at different implementation rates depending on their current capacities and capabilities. For large organizations the transformation might proceed in “waves” for example where the focus is on individual business units.

Check:

Collect transformation data:

• Transformation Metrics. Transformation Backlog items completed by team and overall.

Is the transformation having an impact?

• Operational Performance: Cycle time, throughput, predictability, defect rate.
• Business Agility Metrics: Time-to-market, customer satisfaction, revenue growth.
• Cultural Metrics: Employee engagement, leadership participation, team autonomy, collaboration.

Learn & Adjust:

• Learn and Adjust. Assess progress, challenges, lessons learned from each Transformation Increment. Make adaptations as necessary. We want to see both progress on transformational metrics (completed transformation backlog items by team), and progress on outcomes (operational performance, agile maturity, cultural shifts). Some data can be collected directly from the agile management tools e.g. Jira, other data will be collected via surveys).

The general approach can be summarized as follows:

Key elements summary:

The following sections elaborate these steps in more detail with some examples.

Step 1: Plan

Establish Clear Transformation Goals & Priorities

The organization needs to be able to answer the why for change. What problems are being solved and what is the overarching outcome being sought? Leadership alignment on goals and priorities is an essential first step.

• What is the current state of the organization – major strengths and weaknesses – where do we start?
• What outcomes do we want from the transformation?  What is the “value” of the transformation? What are the objectives? This typically includes things like:
  • Improved Operational Performance: Delivery Cycle Times (Release in weeks vs. months), Predictability, Quality, Productivity, ability to pivot faster to avail of new opportunities.
  • Higher Customer Satisfaction: Superior customer experiences, improved customer growth & retention rates.
  • Better Employee Engagement: Higher levels of autonomy and growth opportunities. Improved retention rates.
• What changes or new capabilities are needed to enable target outcomes?
• How will we measure progress?

For each target outcome, identify the enabling capabilities and practices needed. Also identify KPIs that will be used to measure the effectiveness of any changes. For example:

Major transformation objectives could be formulated as OKRs. OKRs are results-oriented objectives. This could be done on a quarterly basis. The Transformation Backlog is then the set of actions needed to implement the OKRs.

Create a Transformation Backlog

The Transformation Backlog is an ordered list of everything needed to achieve the goals of the transformation. Both Transformation Objectives and Enabling Practices can be captured on a Transformation Backlog. Each item in the Transformation Backlog can be modeled as a ‘story’. All transformation stories should have unique Acceptance Criteria, and there must be a common Definition of Done for all backlog items.

For example:

Story: Agile Delivery Framework Adopted

Acceptance Criteria:

• Team trained in Scrum/Kanban
• Roles in place: PO, SM, Developers
• Team size is 10 or less
• Initial Product Backlog in place, prioritized by PO
• Team has all skills to deliver backlog items to a DOD
• Events & cadences established: Sprint planning, Daily stand-ups, Sprint reviews. Sprint retrospectives
• Work tracked in agile delivery tool: Jira/ADO
• Team has working agreement plus DOD
• Team establishing a stable throughput: at least 2 sprints with > 50% of stories completed to a definition of done.

Definition of Done:

• Agile Coach/ATO verifies story meets Acceptance Criteria. (Evidence attached to each story, e.g. screenshots: team rosters, product backlog screenshot, outlook calendar showing event schedule, Jira Board screenshot, backlog screenshot, DOD, and so on).
• Story accepted as complete by coach.

This might seem like an excessive amount of oversight, but it is essential to bring a degree of objectivity (transparency) into the process to enable Inspection and Adaptation.

Each major Transformation Objective can be modeled as an Epic on the Transformation Backlog. Epics are containers for everything needed to implement the objective. These items can be represented as User Stories with Acceptance Criteria. Teams will implement these Backlog Items and if an agile tool like Jira is used, it should be straightforward to measure progress.

Transformation user stories should be created and maintained by the Agile Transformation Office (ATO). This will ensure that all teams are working towards a consistent set of transformation objectives, and that measured progress will be meaningful. Epics/Stories should be sized to fit in appropriate implementation time-boxes. Large stories (for example Adopt Agile Delivery Framework) might be scoped to fit in a quarter, and split into items based on acceptance criteria) that can be delivered in sprints.

Once an initial Transformation Backlog has been created, this becomes a template for all teams.

Step 2: Organize

Transformation Governance

Lack of effective governance is a major cause of failure in Agile transformations. A transformation governance framework is critical to steer the transformation to a successful outcome.

The word ‘governance’ might feel out of place in the minds of many agile practitioners as it is often associated with rigid control and layers of oversight. In Agile Transformation, however, governance should provide:

• Clear accountability

• Transparency

• Alignment to strategic goals

• Fast feedback loops

• Support for empowered teams

1. Clear Accountability: Roles and Responsibilities

Of course everyone in the organization must contribute to the transformation, however we need a group of people with real ownership and accountability for leading and sustaining the change. This team may be referred to by names like: the Agile Transformation Office (ATO), the Agile Center of Excellence (ACE), or the Lean-Agile Center of Excellence (LACE).

Leadership plays a critical role in any transformation. The leadership team owns the outcomes. They provide strategic oversight and executive sponsorship. They must also play an active role as mentors, coaches, impediment removers/problem solvers, and must invest time and resources accordingly. To do this, they must understand what needs to be done and how to get it done.

 — W. Edwards Deming

 — W. Edwards Deming

Leadership transformation must precede Agile Transformation. Agile initiatives become superficial process changes without leaders who understand and embrace empiricism and self-organization.

There are many ways to organize a transformation, but generally speaking, we need 3 basic roles: business leadership/sponsorship, transformation governance, and transformation delivery teams. We will keep the role names as generic as possible.

• Executive Steering Committee:
  • • Aligns the transformation with business strategy
    • Generates organizational support
    • Participates in planning
    • Empowers teams
    • Removes barriers and impediments
    • Reports progress to the CEO/Leadership Team, highlighting issues and decisions for resolution.
    • Constant communication and encouragement to the broader organization on the vision, strategy and progress. (Shift “Have To” to “Want To”).
• Agile Transformation Office (ATO)
  • • Individuals within the organization who advocate for Agile Transformation.
    • Creates/Maintains the Transformation Backlog.
    • Guides the organization through implementing the Transformation Backlog.
    • Accepts Transformation Backlog items as meeting a Definition of Done
    • Maintains a Transformation Dashboard used to track progress
    • Provides coaching, training and mentorship to teams and leadership.
    • Ensures alignment between Agile practices and business objectives.
    • Identifies roadblocks and helps remove them.
• Agile Delivery Teams
  • • Implements the Transformation Backlog.
    • Works closely with the ATO to implement change.
    • Team Scrum Masters ensure Transformation Backlog items are accounted for in Sprint Planning.

2. Governance Artifacts

3. Governance Events and Cadences

Eating one’s own dog food in matters of governance signifies conviction in the basic pillars of agile delivery; small batches of work delivered in short time boxes, with feedback loops built in to support regular inspection and adaptation. All of which should operate at a sustainable pace to avoid overloading teams and creating excessive stress and fatigue.

Governance of the transformation process is required to ensure a consistent and systematic approach. This includes formalizing accountabilities to ensure that the right people are involved in the transformation process and that there is accountability for outcomes.  Progress can proceed empirically, based on a recurring set of events where evidence or data from recent changes is inspected and the next steps defined based on the results of those inspections. The organizational accountabilities (roles),  evidence needed (data/artifacts), and  events (progress checkpoints/reviews), collectively represent a ‘governance model’.

Each event in the transformation governance framework is a formal opportunity to inspect and adapt transformation artifacts. These events are specifically designed to bring transparency to the state of the transformation. Transparency enables inspection which in turns enables adaptation. Transformation governance events operate on a regular cadence to reduce overhead and complexity.

Governance is provided via a series of events designed to support the following activities:

• Transformation Planning. For example, plan in 90-day increments. Each increment will have a set of objectives (Epics or OKRs) for what is to be accomplished. Inputs to each transformation increment are derived from the transformation backlog. If you are already doing some kind of ‘Quarterly Planning’, or ‘PI Planning’, ART Backlogs should contain transformation activities as deliverables for the next PI.
• Bi-Weekly/Monthly progress checkpoints. These events should be highly data driven – use Burn-Up/Burn-Down charts and dashboards to help visualize progress and identify roadblocks. Make adjustments to plans based on what is learned at each checkpoint.
• Coach Syncs, facilitated by team coach. Review progress vs transformation goals for current iteration/increment. Provide ongoing guidance and help with impediments. For example, a coach from the ATO holds weekly sync with each team to assess progress vs. their transformation goals for the current sprint or planning increment.
• Leadership Syncs, facilitated by ATO. Monthly/weekly progress check. Assist with impediments or issues escalated from the delivery teams. Data from team transformation backlogs is ideally summarized in a dashboard to provide a clear picture of progress.
• Inspect & Adapt Workshops. Review/Demo transformation objectives achieved – for example at end of each 90-day transformation increment.

Here is an example of a set of transformation governance events:

4. Governance Metrics

It is critical to have a measurement system in place as early as possible. Build in metrics for success. Flying blind is not the situation we want to be in. At least not for long. Don’t be one of the many failed transformations simply because you are unable to demonstrate progress, or worse, are unable to see that you are heading in the wrong direction.

Ideally, we will have a transformation dashboard that will provide a transparent picture of the state of the transformation, and can be used to communicate progress to the organization, and also as a basis for making adaptations throughout the journey. An effective Agile Transformation Dashboard should provide clear, actionable insights into both: how Agile practices are being adopted and also whether they’re delivering value. The dashboard should be useful for executives, Agile coaches, and delivery teams, balancing strategic outcomes with team-level progress. It can be leveraged as a primary artifact for all of the transformation governance events.

Getting a good measurement system in place is not a trivial task, and may require significant effort. It is advised to get this underway as early as possible.

Some potential metrics to track:

Step 3: Plan & Execute

Each organizational unit undergoing transformation will likely operate as follows:

Planning. Identify objectives for the next 90-day increment and build the associated transformation activities into their backlogs.

Execute: Teams, or teams-of-teams implement transformation initiatives, for example experimenting with various CI/CD practices and tools. The impact of these actions on transformation KPIs can be measured and tactics adapted as necessary.

Inspect and Adapt. At the conclusion of each increment assess the impact of transformation actions. If any initiative results in no measurable improvement, make sure you understand why, and make any necessary adjustments or adaptations to strategy. This may translate into new items for the backlog for the next increment.

Step 4: Learn and Adapt

Don’t wait until you think you know everything about transformation before you start. Also, don’t copy and paste what has been done elsewhere, even if successfully – it it is unlikely to work in your own context. Agile transformation is a journey of experimenting and learning, and solving your own problems. Your organization, people, culture, challenges and opportunities are unique. Above all, learning should not be an ad hoc process. Learning must be built into the transformation framework at all levels via a built-in set of formal learning events.

Transformation means evolving an organization where people want to constantly improve. To do this we need to provide people with the skills and tools of improvement, and incentivizing them to apply these skills every day. Everyone in the organization must contribute to improvement. Everyone must be trained in improvement, and empowered to improve. This requires new patterns of thinking – a complete mindset change. An agile transformation must not be only about the tactical gains of adopting a set of tools and methods, but must primarily be about re-engineering the culture to where continuous improvement is a way of life. Tools and methods come and go. The goal must be to install a continuous improvement machine within the organization in which everyone plays a part.

Knowledge of agile values, principles and methods is a necessary but insufficient skillset to participate in any successful agile transformation.  Leadership and relationship skills are also necessary at all levels of the organization. The ability to make sound decisions, solve problems, resolve conflict, get buy-in and build consensus is  going to be instrumental in getting to a successful outcome.

Fundamental organizational transformation can be perceived as an existential threat to members of leadership teams, and middle management. All change is accompanied by conflict. Expect it. When we say “conflict” we simply mean different points of view about strategy, tactics, or how to solve various problems that emerge along your journey . Even when the organization is broadly aligned behind an overarching vision, conflict is still likely in areas of strategy and implementation tactics. However if we expect everyone to be a Change Agent, we also need to equip them with some basic skills for leading change, which means working continuously on alignment,  solving problems, resolving conflict, and influencing others.

Conclusions

Agile Transformation need not be a risky adventure through uncharted waters. This article provides a basic framework based on an incremental, iterative approach designed to minimize risks and ensure steady, measurable progress.