Test automation is meant to reduce the cost of testing, improve quality, and shorten time to value. It is possible to achieve all of these. Test automation has never been a silver bullet, though. Many automation efforts fail because of inflated expectations, wrong tools, and poor implementation.
Here’s what you need to do before you start:
- Choose the benefit you desire. You can get savings, quality, and speed, but you will not get all of them at once and none of them will be gained fast.
- Decide how you measure your effort. Use your metrics to lead your effort and improve it.
- Make an implementation plan and follow it. Be prepared to adjust your plan, even radically, when you learn more.
- Build to last. Test automation seldom pays off as a quick & dirty job. Just like in manufacturing, you can reap the benefits of automation only if you produce high enough volume and for a long enough time. Therefore, your test automation must survive high volume and long lifetime.
Don’t start measuring savings, quality improvement, or time to value too early. There is no point measuring them because they are unlikely to be there yet. Start by measuring the time it takes to automate a test case. Then measure the effort you spend on maintaining your automated tests. Those two indicators tell early if you are doing the right thing, if you are doing it right and if you are getting better at what you do. Our senior architect Antti shares his insights on How to select the right KPIs.