Conquering the four levels of test automation

Nov 13, 2018 / by Jani Haapala

The path to becoming a seasoned test automation professional is more about the journey itself than reaching and conquering the final level, says test automation evangelist Jani Haapala. So far, we can see four levels, but other more challenging and enlightening stages may lie in our future.

It’s well known that test automation professionals face many obstacles on the journey from rookies to top-level players in the field. When we select the challenge of becoming test automation experts, many of us start out as manual testers in the hope of one day becoming a test automation developer, exploratory tester or even a requirements specification specialist.

Level 1: The Beast of Automation

Once we accept the challenge of the test automation, we confront the two-headed beast that occupies the first level of the landscape. On this level, players or test automators must grapple with the challenges of automation as well as the tests themselves. Test automation offers the advantages of speed and the ability to replicate en masse. However, as the range of tests increases and as application architectures and solution maintenance become more complex, costs rise, and efficiency begins to fall.

To tame this beast, it’s important to “know the language you’re using” and treat tests as runnable specifications while ensuring that the message and meaning of the tests are understood by your target audience. Of course, it won’t hurt to have a dedicated team of blacksmiths crafting epic swords and shields to developers can use on the quest.

Level 2: The Beast of Mass

As testing professionals continue on their quest, they may not notice the exponential increase in the number of tests they generate. Before long, everyone across the organization gets the means to automate tests in their toolkits while improved solutions accelerate the creation of tests, causing the number of test cases to escalate from 100 to 100,000 in a heartbeat.

Even a failure rate of 1% in 10,000 tests, this means 100 failures, while the time required to debug a single failed test may expand from one to 100 minutes. The result is a situation where an organization may create 100 tests in just three hours but take 20 days to debug 100 failed tests. This in turn leads to the perception that “something is always broken” and masks true visibility into quality.

Faced with such a scenario, testers can select one of two methods to tackle the issue. The “critical red” tool to tackle and defeat the mass monster. Critical red involves quickly identifying and eliminating failures in the testing process. Another resource at the test professional’s disposal on this level is the “jailing test”, an automated process to sort through test failures and tackle the most problematic ones.

Level 3: The Beast of Meaning

The third challenge our evolving test professionals face on their journey is understanding the knowledge pyramid. At the lowest level of the pyramid, they would have realized that they have been accumulating masses of data in the form of test cases. But what is the meaning of all this data? Answering this question forms the basis of the level-three challenge, which is to capture the beast of meaning.

To do this, testers must be able to scale the knowledge pyramid, a quest that requires completing three stages. The first phase involves progressing from raw data to information, which uncovers what the tests reveal about the systems they target.

At the second level, testers progress from using information to acquiring knowledge, where knowledge is the practical application of skills, the isolation of insights and the ability to develop an understanding of trends to proactively prevent certain things from happening. Finally, the attainment of wisdom involves the ability to identify and understand patterns, all of which can be applied in a process of continuous improvement.

Level 4: The Beast of the Future

The final level of the journey sees testers transformed into test automation builders as a result of the challenges they have tackled on the previous levels. But what does this level hold? What new technologies and applications are hidden in the mists of time?

We don’t know if they will present new opportunities or if they will pose new challenges. Or if testers will work with Robotic Process Automation, Artificial Intelligence, or even self-learning systems. Although we cannot yet answer these questions, we can be sure that the journey will continue to be as inspiring and exciting as it was during the early stages of professional development.

Jani Haapala is Delivery Lead at Qentinel. He spends his time conjuring up ways to help customers achieve their goals and supporting Qentinel consultants as they deliver the best solutions to meet customers’ needs. He is a dedicated DevOps advocate and evangelist who has been spreading the word in the software industry for well over a decade.

Topics: Test Automation

Jani Haapala

Written by Jani Haapala