The Agile Approach to Testing

Many organizations have a process that separates implementation and testing into different phases of work, usually carried out by separate teams. Agile processes encourage teams to integrate testing with implementation to shorten the feedback loop (Figure 11-1). Testing takes place continuously, as changes are made. This is done by testers and developers working closely together, combined with automated testing.

Figure 11-1. The change-test-fix feedback loop

So the most useful goal for test automation isn’t to reduce the length of a project’s testing phase but to embed testing and fixing activities into the core workflow. As someone works on a change to the system, whether it’s a change to application code or to infrastructure definitions, they continuously test. They test to make sure their change works as they expect. They test to make sure their change doesn’t break other parts of the system. And they test to make sure they aren’t leaving any loose ends to cause problems later on.

People test so they can fix each problem right away, while they’re still working on the changes. Everything is still fresh in their mind. Because the scope of the changes is very small, problems are quick to find and fix.

Automating Tests for Fast Feedback

Teams whose testing process is based on separate implementation and test phases often attempt to adopt automation by automating their test phase. This is often a project owned by the QA team, which aims to create a comprehensive regression test suite that replicates what they have been doing manually. There are several reasons why these automation efforts tend to be disappointing.

One problem is that automated test suites built by a separate testing team tend to focus on high-level testing. This is because tools marketed at test teams tend to be UI-focused, because they are seen as replacing manual testing driven through the UI.

While UI level tests are an essential part of an automated test suite, they are more expensive to develop, slower to run, and require more work to maintain the systems they test evolve. So a test suite needs to be well balanced. The test pyramid described later in this chapter offers a model for designing a well-balanced test suite.

However, a key to successful test automation is for the entire team to be involved in planning, designing, and implementing it, rather than making it the responsibility of an isolated group.

THE PITFALL OF BIG BANG TEST AUTOMATION PROJECTS

Initiatives to implement automated testing often bite off more than they can chew. This leads to difficulties keeping up with ongoing development because the system is a constantly moving and changing target. Before the massive test suite is complete, the system has changed. Once the test suite is built, the system will immediately change again. So tests tend to be constantly broken, and the nirvana of a complete test suite is never achieved.

Aiming for the goal of a complete, finished test suite is doomed to fail. Instead, the goal of an automation initiative should be to embed the habit of continuously writing tests as a part of routine changes and implementation work. So the outcome of an automated testing initiative is not a completed test suite, but a set of working habits and routines.

When automated testing has been successfully adopted by a team, tests are written or updated whenever a change is made to the system. CI and CD regimes run the relevant tests for every change continuously, as discussed in Chapter 10. The team responds immediately to fix failing tests.

Organically Building a Test Suite

The way to start an initiative that results in good testing habits is not to attempt to build a full test suite to cover the existing functionality. Instead, write tests for each new change as it comes up. When a bug is found, write a test that exposes that bug, and then fix it. When a new feature or capability is needed, begin implementing tests as you go, possibly even using test-driven development (TDD), as described later in this chapter.

Building the test suite organically as a part of making routine changes forces everyone to learn the habits and skills of sustainable, continuous testing. Again, the outcome to aim for is not a “finished” test suite, but the routine of testing each change. A test suite will emerge from this approach, almost as a by-product. Interestingly, the test suite that emerges will be focused on the areas of the system that most need tests: the ones that change and/or break the most.