The Perils of Automation Outside of a Sprint

Picture the scene. You are a tester/QA/SDET in an Agile team. Releases are scheduled at the end of each sprint. Everything is going swimmingly until you hit a blip. It could be a change where writing the automated test difficult is, or there is an urgent change/fix needed late in the sprint and everyone pivots to it, or you could even just be on holiday (key man dependency? Never...).

Either way, you just can't get the work fully done by the end of the Sprint. So, what happens next?

I have seen a few possible choices in my career, but they generally fall into 3 main camps:

1. The story is carried over It's just the same as if the development work wasn't completed, if the testing activities aren't, the story isn't Done. So it carries over to the next sprint.
   

2. The outstanding work is decoupled The development has been completed, and it has been verified manually, but the automation isn't done. The team decides to deploy the code and the automation task is split into a new ticket and carried over into the new sprint.
   

3. The outstanding work is added to a separate backlog In this case, there is a separate backlog for automation work. If it can be done before release, great. If not, it falls into a different backlog and is managed separately. Either way, the code is released.

Now, each of these cases have pros and cons.

In the first case:

• Code is not released, and value isn't realised: Con

• The tests for new functionality won't be skipped: Pro

• The team have something concrete to discuss in the Retro: Pro

In the second case:

• The value in the code is realised: Pro

• There are no automated tests in place: Con

• The work can be factored into the next sprint: Pro

• The team have something concrete to discuss in the Retro: Pro

And in the third case:

• The value in the code is realised: Pro

• There are no automated tests in place: Con

• The work has disappeared from the sprint: Con

• It may come up in the Retro unless this behaviour is normalised: Con

The big danger, I feel, is the third case. I have had experience with a number of teams where the "Automation Backlog" is the normal way of working. The main focus of the teams is the development work and shipping code. In some cases, the majority of automation tasks fall into a backlog.

Whether the people doing the work are in the sprint team or outside, there is a disconnect. You can have people working on two backlogs, which can cause priority conflicts, or you have an external team picking up the work, who won't be as close to the detail or have access to the original team for questions/queries.

I have also seen the work in the automation backlog start to creep up, especially when it supports multiple teams. As a result, there can be regression errors slipping through. You can also have a number of similar changes to have tests written, so some could be outdated by the time they are worked on.

So in summary, try to keep your testing in your teams. It saves on real headaches!