Kanban for Software Teams

Summary

• Combined with Flow Metrics, a Kanban System is a highly effective method for agile software development.
• Kanban is a practical, evolutionary approach to optimize workflows, improve predictability, and reduce development cycle times.
• A Kanban System is framework-agnostic, and can be used either stand-alone or to supplement other agile frameworks like Scrum.
• This article covers how to setup, operate and continuously improve a Kanban System for software development.

The Kanban Method for Software Development

The Kanban Method is a visual workflow management approach that helps software development teams improve efficiency, optimize flow, and continuously deliver value. It is based on Lean principles and emphasizes limiting work in progress (WIP), eliminating waste, and continuous improvement.

Lean provides the overarching principles on which the Kanban Method is based:

• Customer Centricity: Define Value from the Customer’s Perspective.
• Identify and Map the Value Stream: Visualize the sequence of steps used to create value for the customer. Use the Value Stream Map to expose and resolve delays, gaps and other sources of waste in your delivery system.
• Create Flow: Make the value-creating steps flow. Work items should move through the workflow without delays or friction, such as handoffs, waiting for approvals or resolving dependencies. Cross-functional product teams better enable flow vs. organizational silos.
• Establish Pull: Only start new tasks when there is available capacity to do the work. Avoid multi-tasking: finish work on existing in-process items before starting new ones. Processes indicate capacity availability through WIP limits.
• Pursue Perfection: Continuous Improvement of flow and pull through Kaizen.

These principles interact in a continuous feedback loop where efforts to get value to flow faster always expose hidden waste and new opportunities for improvement in the value stream.

There is no end to the process of reducing cycle time, cost, defects and waste in the quest to provide increasing value to your customers.

Eliminating waste and improving flow are the primary focus of the Lean continuous improvement methodology.  To achieve these goals, Lean uses a number of tools, including:

Lean production methods including Kanban have been around for decades. They are widely acknowledged as a highly effective methodology for improving productivity, and decreasing costs in manufacturing organizations.

More recently software development teams have been interested in flow-based delivery methods. The Kanban Method is a workflow management system designed to visualize work, limit work-in-progress (WIP), and improve flow. Kanban is framework-agnostic, and can be used as either a stand-alone development method or to supplement other agile frameworks like Scrum to improve the flow of work through delivery processes. Kanban is a visual, WIP-constrained pull system. A Pull System allows teams to start new work when they have capacity instead of others pushing work onto them regardless of their current workload.

Kanban aims to balance demand and capacity, enabling teams to deliver a continuous flow of value with greater predictability. The heart of Kanban lies in visualizing the delivery system workflow so teams can proactively manage and improve it to achieve optimal cycle times. When used as a stand-alone delivery system for software development, teams frequently supplement Kanban with additional practices borrowed from agile frameworks, such as stand-ups, intake refinement meetings, and retrospectives. Similarly, Scrum teams can adopt Kanban concepts, like improved visualization and applying WIP limits, to enhance transparency and boost throughput.

There is an interesting end-note in the Scrum Guide: The Scrum framework, as outlined herein, is immutable. While implementing only parts of Scrum is possible, the result is not Scrum. Scrum exists only in its entirety and functions well as a container for other techniques, methodologies, and practices. Likewise, leaving out any of the 5 principles from Lean sabotages your agility.

Kanban Implementation Strategy

The process of implementing Kanban is divided into a number of steps: initial set up of the system, operating the system and continuously improving the system. Each step has different goals and activities. Setting up a Kanban system involves designing the workflow, setting WIP limits and workflow policies, and aligning on the data and events needed to operate and improve the system. This is the foundational step that lays the groundwork for ongoing operations.

There are a number of basic principles used in setting up and operating a Kanban System.

1. Start with what you do now. Kanban does not prescribe major process changes upfront. Teams map their existing workflow and improve it gradually through Kaizen.
2. Respect current roles, and responsibilities. Unlike Scrum, Kanban does not require new roles (e.g., Scrum Master, Product Owner). It works with existing roles and team structures.
3. Agree to pursue incremental, evolutionary change. Change is based on continuous, small, data-driven improvements (Kaizen) without major disruptions to current processes. 

Kanban adoption requires no major disruption to existing delivery processes or organizational structures. Kanban is framework-agnostic. The transition can be evolutionary and self-directed. Here is a summary of the basic steps for setting up a Kanban System for software development:

Kanban System Setup

Goals of the Setup Phase:

• Create a system that reflects the team’s current workflow.
• Establish a shared understanding of how the system works.
• Ensure team readiness to operate the system.

Step 1: Align Team and Stakeholders on the Meaning of “Customer Value”

In Lean methodology, “value” is defined as everything that a customer is willing to pay for; meaning, any characteristic or feature of a product or service that a customer considers essential to meet their needs. Anything that does not provide value to the customer is considered waste. 

A ‘Voice of the Customer’ (VOC) initiative based on customer interviews or surveys can help identify what customers value most. A team’s goal must be to create products and services that align with customer’s needs and expectations. The Voice of the Customer should be clearly reflected in the Product Backlog. The ranking of items in the backlog should be based on value to the customer.

Step 2: Visualize Your Value Creation Workflow.

Every business has a set of processes that are used to bring a product or service to their customers. In lean this is referred to as a value stream. A value stream is the series of processing steps necessary to transform a customer request into a deliverable item of value. For software, a typical sequence includes steps like: Backlog Refinement, Analysis, Development, Testing, and Acceptance. The method used to implement this step is Value Stream Mapping, a tool from Lean to visualize the sequence of steps involved in the value creation process.

Teams use a Kanban Board to render their work visible. The individual steps in the workflow are represented by columns on a board (either physical or electronic). The details of work items are captured on cards which are then placed in the column representing their current state. For software development, cards typically represent User Stories, though other artifacts such as Features or entire initiatives (Epics) may also be represented. Bringing clarity and transparency to how the work gets done is a prerequisite for problem-solving and meaningful change – you can’t fix problems that you can’t see. Transparency promotes a common understanding of the work between members of a delivery team, and establishes an objective basis for enacting change.

Split each state in the workflow (except the end states), into a Doing and Done state. The Done state acts as a buffer state for completed work to wait until capacity is available in the next workflow state. This step is important because it makes it possible to identify and react to emerging congestion within the system. This makes idle work visible, and enables rapid diagnosis of flow-related problems.

This also makes it easier to measure and focus on flow efficiency, that is, the relative amount of time work spends waiting to be processed vs. actually being worked on. Buffer states are considered Non-Value-Added states, which means they increase the Work In Progress (WIP) of your flow – each extra item of WIP carries a penalty in terms of cycle time (per Little’s Law). A Kanban Team should always be concerned with improving their flow efficiency. Think of a workflow as a pipeline that produces value. Anything that slows down the pipeline is waste. The goal to is to increase value by reducing waste and continuously improving the workflow.

Step 3: Apply WIP limits to each step of the workflow – Implement Pull

Setting Work In Progress (WIP) limits ensures teams focus on completing existing work items before starting new ones. Setting WIP limits on different stages of a workflow creates a “pull” system, meaning that new work is only pulled into a workflow stage when there is capacity available. This prevents overloading workflow steps and enables smoother flow through the system with stable, predictable cycle times. This is a core principle of the Kanban methodology.

WIP-limited pull is the central flow-control mechanism in Kanban. Work-in-Process is the accumulation of work within or between processes. Limiting WIP means setting a cap on the number of work items that can be worked on simultaneously in any stage of the workflow. This practice prevents overloading the team and ensures a focus on completing tasks before starting new ones. This prevents queues of work from building up between steps, keeps the cycle time for each step to a minimum, and improves overall flow efficiency – that is, ensures more time is spent actively processing work items vs. waiting in the system.

When WIP within each process step is minimized, end-to-end cycle time (and processing cost) is minimized. This relationship is enshrined in Little’s Law:

This may seem like common sense, viz. the time to get a work item through a process is the amount of work waiting to be processed divided by the processing rate per item. The amount of WIP between each step in a workflow is effectively a queue. The longer the queue the longer an item must wait to be processed, directly causing longer cycle times (and increased processing costs) per Little’s Law.

The end-to-end Cycle Time through the entire system is the sum of the individual process cycle times. To achieve our goal of minimizing Cycle Time per backlog item, we constrain the length of the queues between processing steps by applying WIP limits. Members of delivery teams can help achieve this goal by committing to finishing work in progress over starting new work. In an ideal system, work items arriving in the ‘Done’ buffer state of each process step would be consumed immediately by the next state with zero or minimum wait time.  The result is that work flows smoothly through the system, improving end-to-end performance.

For example, a development team might use pull principles to manage tasks through a Kanban board, ensuring no one takes on more work than they can handle. Simultaneously, they optimize flow by reducing handoff delays, automating testing, addressing bottlenecks and blocked work items.

By combining Flow and Pull, software teams can achieve faster product delivery while remaining adaptable to changing priorities.

The optimum WIP level for any processing step is a reflection of that step’s capacity to process work. We neither want to overload it’s capacity or starve it of work either. Set WIP limits to some initial values (say based on the number of team members supporting each processing step) and then refine or adjust over time. Measure the impact of each adjustment on process Cycle Time. Ultimately we want to achieve a balance in the flow between adjacent processes where the average rate of output from one process matches the average rate at which the adjacent downstream process can consume that work.

In a manufacturing environment where the product being produced has little variation (unlike software), these limits can be calculated mathematically to achieve balance between processing steps. For software development where every new feature or user story is unique, this problem must be solved empirically. WIP constraints are the most effective way to achieve rate-matching between adjacent processes.

Remember, the more work we start the less we finish. Pushing more work into the system without completing any work dilutes our ability to get things done. More unfinished work in the system means more context switching, and more administration overhead, all of which slows down our speed of delivery. This is the purpose of WIP limits.

Also worth emphasizing:

• Without WIP limits, it is impossible to have a stable, predictable delivery system.
• Without WIP limits, you don’t have a Kanban system.

Step 4: Define ‘done’ for each step in the process.

Being clear and explicit about what ‘done’ means for each step of the workflow further enhances overall transparency about how the system operates, and provides a foundation for how to improve it. A transparent definition of done brings predictability and ensures quality. Making the exit criteria explicit means that the work for each step is done in a consistent and repeatable way, and that the work item is in a known state when done.

One of the core concepts in Lean Manufacturing is Standard Work. The idea is that we get consistent results from a process that is applied consistently. This may seem anathema for knowledge workers who are motivated by autonomy. The idea isn’t to use the standards as a way to limit the team. Rather, the standard is to be used as a baseline for continuous improvement.  It is only possible to evaluate improvements objectively when existing workflow procedures are ‘standardized’. As standards improve, the new standard becomes the basis for further improvements: improving standardized work is a never-ending process.

Now our pull system is constrained by 2 things: only pull an item when there is capacity to work on it (WIP constraint), and when the item meets the definition of done for the previous step in the workflow (workflow policy constraint).

Operating the Kanban System

Goals for Operating the System:

• Identify and address inefficiencies and congestion in the workflow.
• Foster a culture of collaboration and continuous improvement.
• Deliver value consistently and predictably.

Proactively Manage the Flow and Deal with Congestion

Meet daily to identify bottlenecks, emergent queues and other impediments or delays to the flow of work. In particular, check if any step is overloaded or if any step is starved of work, or if any items are stuck in a step well beyond their normal cycle times. For the latter issue some tools will display the age of the item in that state. Stand-ups should be focused on maintaining the flow of work (congestion control). What items are late (beyond their expected cycle time), or at risk of being late? Priority should always be given to finishing work versus starting work. If necessary make adjustments to improve the flow.

Maintaining the flow means actively managing all work items in progress between the Backlog and Done states. A Kanban development team should meet on a regular cadence (for example, conduct a Daily Stand-up) to review their board and agree what adjustments need to be made to address issues of flow control and congestion. In the above example what action should the team take?

Managing the Flow: The Kanban Standup

A Daily Stand-Up for a Kanban team is the event where a team actively manages the flow of work items in their system. The event takes place around the Kanban Board and seeks to identify steps in the workflow that are overloaded, starved of work, and on what actions the team can take to recover the flow.

Active management of items in a workflow can include:

• Controlling WIP: limiting the number of items that are being worked on simultaneously – don’t pull any new work into a processing step when WIP reaches its upper limit. (Send items back that breached a WIP limit, and focus on getting items done before starting new ones).
• Address work items are aging beyond expected Cycle Times, and preventing work items piling up in any part of the workflow – each of these items should have a specific owner and action plan.
• Unblocking blocked work (taking action, including escalation beyond the team when necessary).

Work Item Aging and WIP are leading indicators and these are the measures that teams should focus on daily in order to proactively manage the flow of work.

Cycle Time and Throughput are lagging indicators (the data is based on work that has already been completed). These are historical data which can be input to Monte Carlo and applied for planning and forecasting purposes. Cycle Time addresses: When will it be done, and Throughput addresses: How much can we do. In both cases, the measures are expressed as a range plus a probability.

Overall, we want to minimize the amount of time an item spends in the system – Cycle Time. A Kanban Standup takes the form of Walking The Board vs. the usual round-robin status-taking. In Walking The Board, the board is reviewed one column at a time, from right to left, along the workflow. In progress items in the state closest to the done state are reviewed first – these represent items with shortest path to done, and hence of the highest economic opportunity. Items in each column can be reviewed in order of item age. This approach also has the benefit of freeing up slots for upstream work items.

The Daily Standup can be driven from a Work Item Age Chart, where the age of each in progress work item is shown versus workflow state and also versus the team’s historical expected time in progress. For example:

A team’s SLE is based on historical cycle time performance data, an example of which might be: 85% of work items are done in 15 days or less.

With this approach, where the focus is on delayed items or at risk items,  we can get through a standup more quickly and effectively.

Measure and Improve Relentlessly: Kaizen

When we setup our Kanban system an initial step was to map out and visualize the existing workflow. If done properly this should quickly expose any obvious gaps, redundancies and other major sources of waste in the system.  Eliminating waste and improving flow are the primary focus of continuous improvement activities for Kanban teams. These improvements contribute directly to business and customer value by reducing delays, improving quality, and lowering costs. That’s a pretty good definition of value:

While the Daily Standup event is focused on making tactical adjustments to maintain the flow on a day-to-day basis, more systemic improvements of the overall workflow need a broader perspective on how the system is performing. The goal is to continuously improve the workflow to deliver greater value, faster, at lower cost. We do this by measuring the overall value stream performance, and using the data to identify sources of delay and waste as work flows through the system. Using a Kaizen approach, change is introduced gradually to improve the process based on the problems found.

In Kanban, a small number of process metrics is sufficient to evaluate the performance of a delivery system. Since the focus of Kanban is the reduction of Cycle Time by improving the flow of work through a system, the metrics used to measure flow performance are flow metrics, Specifically, Cycle Time, Throughput and WIP.

The relationship between these metrics is described by Little’s Law, and the Continuous Flow Diagram (CFD) is a good way to visualize these relationships. The CFD shows the distribution of work in each of the workflow states and how that distribution is trending over time. Each band in the diagram shows the number of work items in each state of the workflow. The CFD Diagram is one of the primary artifacts of the Kanban Method and can be used to visualize bottlenecks and uneven workloads in the value stream. The CFD can be used to measure flow performance – Cycle Time, Throughput and WIP –  and help identify problems and opportunities for improvement.

Reading a CFD can provide us with each of the fundamental measurements of flow, namely Cycle Time, Throughput and WIP. One can see from the above example CFD chart that after day 7 the work arrival rate has increased while throughput has been unable to keep up. This has caused an increase in WIP, and a consequent increase in Cycle Time. Anytime the arrival rate exceeds the departure rate (or throughput), WIP will accumulate and Cycle Times will increase per Little’s Law. The CFD is a powerful tool for diagnosing problems in a system. This data provides the team with the data needed to decide what actions are needed to improve Cycle Time. The relative priority of all proposed improvement initiatives should be directly proportional to their impact on reducing Cycle Time.

Another useful tool in bringing focus to Cycle Time improvement is the Cycle Time Histogram. This is a chart showing the distribution of delivery cycle times (for example for User Story cycle time in days) by frequency. Cycle Time is the time from development starts to done.

This is useful (and actionable) data for a team that has a goals like:

• deliver majority (>50%) of user stories in 2 days or less, and,
• deliver 95% of all stories in under 10 days.

The Agile Retrospective event provides a regular opportunity for reviewing delivery performance (Throughput, Cycle Times and WIP levels). The basic flow metrics need to be collected and managed to support improvement. The data needs to be readily available for the retrospective. This data can help a teams understand how well their value stream is performing, and to identify patterns, relationships, and opportunities for improvement.

More on Flow Metrics here. More on retrospectives here.

Improvement Strategies

Some of the more obvious targets for improvement actions include items from the list below. Teams are encouraged to experiment with these using the Plan-Do-Check-Act (PDCA) framework, and measure their impact on their key flow metrics:

Conclusions

The Kanban Method brings strong focus on accelerating delivery cycle times. It can be used as a standalone method or complement other agile frameworks like Scrum:

• Visualize your workflow in a way that enables diagnosis of flow-related problems.
• Limit the amount of work pulled into each processing step to minimize queues and delays.
• Continuously measure and improve.

Like Scrum, the Kanban Method is intended to be a team sport, and improvement is best achieved through inspection and adaptation based on transparency and collaboration. Some Scrum teams may wish to relinquish their fixed time-boxes and try a continuous flow type development model. They should by all means give it a try.