Effective Retrospectives

“Having no problems is the biggest problem of all”

– Taiichi Ohno

“A little rudder early is better than a lot of rudder late”

Don Reinertsen

Continuous improvement is at the heart of agile software development and is one of the pillars of lean. For those practicing scrum, the retrospective exercise gives teams the opportunity to inspect and adapt at the end of each iteration, and thereby improve their effectiveness over time. Many teams however seem to get stuck with an approach that borders on the superficial and which leads to little meaningful improvement.

Frequently, teams take an approach where they capture their observations about the previous iteration into categories like:

  • Keep Doing
  • Stop Doing
  • Start Doing

An example of this approach might produce something like:

A Retrospective Readout
A Retrospective Readout

In this example, the team clearly has some problems in meeting their commitments on stories, velocity and quality. However it also looks like they concluded that a combination of better unit testing and additional QA resources would be the answer to these problems. (Never mind the fact they dumped a substantial amount of blame on the QA team!). The team may have jumped to these conclusions in an arbitrary way without using any basic problem-solving tools or improvement methodology. Fundamental to informed decision-making is having data and measurements on the issue that needs to be addressed. Let’s take a look at some data the team might have reviewed as part of their retrospective.

First, the burndown chart for the previous iteration:

Iteration Burndown Chart
Iteration Burndown Chart

The team only completed 20 of the planned 30 story points for this specific iteration. Now let’s look at the defect status:

Iteration Defects
Iteration Defects

The team exited the iteration with a total of 6 open defects, 3 of which were of major severity. Let’s focus on the open defects. The first thing we need to do is get to a root cause for why there were any open defects. There are a number of tools or techniques for doing this such as the ‘Five Why’s’ exercise, or via a Fishbone Diagram. Five Why’s is a simple technique that helps a team quickly get to the underlying causes of a problem. The technique comprises asking ‘Why?’ five times about a problem. Usually by the fifth Why, we will be at, or very close to, the root cause of the problem. Let’s try it on the open defects problem:

Q1: Why were there open defects at the end of the iteration?

A1: Because there was not enough time to fix them.

Q2: Why was there not enough time to fix them?

A2: Because they were found in the last few days of the iteration.

Q3: Why were they found in the last few days of the iteration?

A3: Because testing for those stories started in the last few days of the iteration

Q4: Why did testing start so late?

A4: Because the stories were not completed by the developers until then

Q5: Why were the stories not completed by development until then?

A5: Because the stories were quite large (13 points).

So now it looks like that at least one of the contributing causes for open defects at the end of an iteration is related to the fact that large stories do not get completed in time to allow sufficient time for testing and bug fixing. The QA team gets pressurized at the end of the iteration to complete testing quickly for these stories. When defects are found, developers do not have enough time to turn around fixes and have them verified. Another valuable data point for the team to examine would have been a Cumulative Flow Diagram for the sprint. For example:

Cumulative Flow Diagram
Cumulative Flow Diagram


The numbers on the bars indicate the number of story points allocated in each stage of the workflow. In this case we can see that at any given time most of the work is happening in the development part of the workflow and that the QA team is waiting for work to arrive in their queue. Thus overall end-to-end throughput is impacted by the imbalance of work between the adjacent process steps of development and test.

The next stage in the discussion would be to generate ideas for addressing this problem. The team’s original proposal to add more testers is not going to help here. The real challenge for the team will be to create smaller stories, using perhaps using alternative story-splitting or slicing approaches or that they may need to learn. They may also need to look at working on a fewer number of stories at a time by limiting WIP.

Many agile teams may generate improvement actions based on perceived symptoms of problems due to the lack of focus, data and structure in their retrospectives. Basic problem-solving tools like Five Why’s and Fishbone Diagrams, combined with solid data, provide deeper insight into problems and can lead to measurable improvements in team performance. The basic process flow of gathering data, identifying problems, understanding root causes and then proposing actions to be taken is well established and is reflected in the Deming Cycle (PDCA) or Six Sigma (DMAIC) methodologies. Teams are encouraged to make small incremental improvements from iteration to iteration rather than taking on too much and not succeed. Experimentation with different approaches and solutions will eventually pay off.