Kanban for Software Teams

Implementation of a Kanban System can be an effective strategy for improving the economics of product development. 

Summary

• A Kanban System can act as a flow accelerator for software delivery.
• Building a Kanban System is achieved via 3 basic steps:
  • Defining & Visualizing the Workflow
  • Actively managing the flow of work through the workflow.
  • Working to continuously improve the flow.
• A Kanban System is framework-agnostic, and can be used either stand-alone or to supplement other frameworks like Scrum.

Although Kanban Systems have their roots in manufacturing, and are often adopted by product support teams, there is increasing interest within software development teams seeking delivery methods with less overhead than Scrum, or, by teams looking to supplement Scrum and other frameworks with additional techniques to accelerate the flow of value through their delivery processes. Kanban is a visual, WIP-constrained pull system. A Pull System enables teams to start new work when they have capacity instead of work being pushed onto them regardless of their current workload. The goal of Kanban is to achieve balance between demand and capacity, and in doing so, to to deliver a continuous flow of value with greater predictability. The heart of Kanban is visualization of workflow so it can be proactively managed to achieve optimal cycle times. When used a stand-alone delivery system for software development, Kanban is frequently supplemented with additional practices borrowed from agile frameworks like Scrum, such as stand-ups, intake refinement meetings and retrospectives. Likewise, Scrum teams can take advantage of Kanban concepts such as improved visualization and the application of WIP limits as a way to improve transparency, and boost throughput.

Kanban adoption requires no major disruption to existing delivery processes or organizational structures. Kanban is framework-agnostic. The transition can be evolutionary and self-directed. Here is a summary of the basic steps for setting up and operating a Kanban system for software development:

Step 1: Define your workflow.

Every business has a set of processes that are used to bring a product or service to their customers. A workflow is the series of processing steps necessary to transform a customer request into a deliverable item of value. For software, a typical sequence includes steps like: Analysis, Development, Testing, and Acceptance. The individual steps in the workflow are represented by columns on a board (either physical or electronic). The details of work items are captured on cards which are then placed in the column representing their current state. For software development, cards typically represent User Stories, though other artifacts such as Features or entire initiatives (Epics). Bringing clarity and transparency to the work is a prerequisite for problem-solving and meaningful change – you can’t fix problems that you can’t see. Transparency promotes a common understanding of the work between members of a delivery team, and establishes an objective basis for enacting change.

Split each state in the workflow (except the end states), into a Doing and Done state. The Done state acts as a buffer for completed work to wait until capacity is available in the next workflow state. This step is important because it makes it possible to identify and react to emergent queues or bottlenecks within the system. This makes idle work visible, and enables rapid diagnosis of flow-related problems. 

This also makes it easier to measure and focus on flow efficiency, that is, the relative amount of time work spends waiting to be processed vs. actually being worked on. Buffer states are considered Non-Value-Added states, which means they increase the Work In Progress (WIP) of your flow – each extra item of WIP carries a penalty in terms of cycle time (per Little’s Law). A Kanban Team should always be concerned with improving their flow efficiency. Think of a workflow as a pipeline that produces value. Anything that slows down the pipeline is waste. The goal to is to increase value and reduce waste by continuously improving the workflow.

Step 2: Apply WIP limits to each step of the workflow.

WIP-limited pull is the central flow-control mechanism in Kanban. WIP Limits are used to ensure that any processing step in a workflow is not being overwhelmed by consuming more work than it has the capacity to process. This prevents queues of work from building up between steps , keeps the cycle time for each step to a minimum, and improves overall flow efficiency – that is, ensures more time is spent actively processing a work item vs. waiting in the system. When WIP within each process step is minimized, end-to-end cycle time (and processing cost) is minimized. This relationship is enshrined in Little’s Law:

This may seem like common sense, viz. the time to get a work item through a process is the amount of work waiting to be processed divided by the processing rate per item. The amount of WIP between each step in a workflow is effectively a queue. The longer the queue the longer an item must wait to be processed, directly causing longer lead or cycle times (and increased processing costs) per Little’s Law. Each extra item of WIP carries a Cycle Time penalty. The end-to-end Cycle Time through the entire system is the sum of the individual process cycle times. To achieve our goal of minimizing Cycle Time per backlog item, we constrain the length of the queues between processing steps by applying WIP limits. One approach to achieve this is to adhere to the principle: prioritize finishing work in progress over starting new work. In an ideal system, work items arriving in  the ‘ready’ buffer state of each process step would be become ‘in progress’ with zero or minimum wait time.  The result is that work flows smoothly through the system, improving end-to-end performance.

The optimum WIP level for any processing step is a reflection of that step’s capacity to process work. We neither want to overload it’s capacity or starve it of work either. WIP limits should be set to some initial values (say based on the number of team members within each processing step) and then refined or adjusted over time. Ultimately we want to achieve a balance in the flow between adjacent processes where the average rate of output from one process matches the average rate at which the adjacent downstream process can consume that work. In a manufacturing environment where the product being produced has little variation (unlike software), these limits can be calculated mathematically to achieve balance between processing steps. For software development where every new feature or user story is unique, this problem must be solved empirically. WIP constraints are the most effective way to achieve rate-matching between adjacent processes.

Remember, the more work we start the less we finish. Pushing more work into the system without completing any work dilutes our ability to get things done. More unfinished work in the system means more context switching, and more administration overhead, all of which slows down our speed of delivery. This is the purpose of WIP limits.

Step 3: Define ‘done’ for each step in the process.

Being clear and explicit about what ‘done’ means for each step of the workflow further enhances overall transparency about how the system operates, and provides a foundation for how to improve it. A transparent definition of done brings predictability and ensures quality. Making the exit criteria explicit means that the work for each step is done in a consistent and repeatable way, and that the work item is in a known state when done.

One of the core concepts in Lean Manufacturing is Standard Work. The idea is that we get consistent results from a process that is applied consistently. This may seem anathema for knowledge workers who are motivated by autonomy. The idea isn’t to use the standards as a way to limit the team. Rather, the standard is to be used a baseline for continuous improvement.  It is only possible to evaluate improvements objectively when existing workflow procedures are ‘standardized’. As standards improve, the new standard becomes the basis for further improvements: improving standardized work is a never-ending process.

Now our pull system is constrained by 2 things: only pull an item when there is capacity to work on it (WIP constraint), and when the item meets the definition of done for the last step in the workflow (workflow policy constraint).

Step 4: Manage The Flow.

Meet daily to identify bottlenecks, emergent queues and other impediments or delays to the flow of work. In particular, check if any step is overloaded or if any step is waiting for work, or if any items are stuck in a step well beyond their normal cycle times. For the latter issue some tools will display the age of the item in that state. Stand-ups should be focused on maintaining the flow of work. What items are late (beyond their expected cycle time), or at risk of being late? Priority should always be given to finishing work versus starting work. If necessary make adjustments to improve the flow.

Maintaining the flow means actively managing all work items in progress between the Backlog and Done states. A Kanban development team should meet on a regular cadence (for example, conduct a Daily Stand-up) to review their board and agree what adjustments need to be made. In the above example what action should the team take?

The Kanban Standup

A Daily Stand-Up for a Kanban team is the event where a team actively manages items in their workflow. The event takes place around the Kanban Board and seeks to identify steps in the workflow that are overloaded, starved of work, and on what actions the team can take to recover the flow.

Active management of items in a workflow can include:

• Controlling WIP: limiting the number of items that are being worked on simultaneously – don’t pull any new work into a processing step when WIP reaches its upper limit
• Finishing items before starting new ones – ensuring work items are not aging beyond expected Cycle Times, and preventing work items piling up in any part of the workflow.
• Unblocking blocked work (taking action, including escalation beyond the team when necessary).
• Ensuring our work items are relatively small, while still providing value.

Overall, we want to minimize the amount of time an item spends in the system – Cycle Time. A Kanban Standup takes the form of Walking The Board vs. the usual round-robin status-taking. In Walking The Board, the board is reviewed one column at a time, from right to left, along the workflow. In progress items in the state closest to the done state are reviewed first – these represent items with shortest path to done, and hence of the highest economic opportunity. Items in each column can be reviewed in order of item age. This approach also has the benefit of freeing up slots for upstream work items.

The Daily Standup can be driven from a Work Item Age Chart, where the age of each in progress work item is shown versus workflow state and also versus the team’s historical expected time in progress. For example:

A team’s SLE is based on historical cycle time performance data, an example of which might be: 85% of work items are done in 15 days or less.

With this approach, where the focus is on delayed items or at risk items,  we can get through a standup more quickly and effectively.

Step 5: Measure and Improve.

While the Daily Standup event is focused on making tactical adjustments to maintain the flow on a day-to-day basis, more systemic improvements of the overall workflow need a broader perspective on how the system is performing. Continuous measurement and improvement of the development workflow is necessary to improve lead times and reduce costs.  Teams must continually improve their processes by eliminating non-value adding tasks and by investing in training, better tools, and automation.

In Kanban, a small number of process metrics is sufficient to evaluate the performance of core delivery processes, and to point at where improvement is needed. Since the focus of Kanban is the reduction of Lead Time by improving the flow of work through a system, the metrics used to measure flow performance are flow metrics, Specifically, Lead Time, Throughput and WIP.

The relationship between these metrics is described by Little’s Law, and the Continuous Flow Diagram (CFD) is a good way to visualize these relationships. The CFD Diagram is one of the primary artifacts of the Kanban Method, and is used to measure flow performance, identify bottlenecks in the process, and to help identify opportunities for improvement. The CFD shows the distribution of work in each of the workflow states and how that distribution is trending over time. Each band in the diagram shows the number of work items in each state of the workflow.

Reading a CFD can provide us with each of the fundamental measurements of flow, namely Lead Time, Throughput and WIP. One can see from the above example CFD chart that after day 7 the work arrival rate has increased while throughput has been unable to keep up. This has caused an increase in WIP, and a consequent increase in Lead Time. Anytime the arrival rate exceeds the departure rate (or throughput – the processing rate per item), WIP will accumulate and Lead Times will increase per Little’s Law. The CFD is a powerful tool for diagnosing problems in a system. This data provides the team with the data needed to decide what actions are needed to improve Lead Time. The relative priority of all proposed improvement initiatives should be directly proportional to their impact on reducing lead time.

Another useful tool in bringing focus to Cycle Time improvement is the Cycle Time Histogram. This is a chart showing the distribution of delivery cycle times (for example for User Story cycle time in days) by frequency. Cycle Time is the time from development starts to done.

This is useful (and actionable) data for a team that has a goals like:

• deliver majority (>50%) of user stories in 2 days or less, and,
• deliver 95% of all stories in under 10 days.

More on Flow Metrics here.

Summary

The Kanban Method can be an effective alternative to Scrum when applied in the right way:

• Visualize your workflow in a way that enables diagnosis of flow-related problems.
• Limit the amount of work pulled into each processing step to minimize queues and delays.
• Continuously measure and improve.

Like Scrum, the Kanban Method is intended to be a team sport, and improvement is best achieved through inspection and adaptation based on transparency and collaboration. Some Scrum teams may wish to relinquish their fixed time-boxes and try a continuous flow type development model. They should by all means give it a try.