This is why you need to test from coding to production

Apr 15, 2019 / by Esko Hannula

Software testing is the oddball in information system development. It is generally easy to estimate how long it will take and how much it will cost. Testing is also widely acknowledged to be the most important bottleneck in the software release process.

In traditional IT projects, serious testing began shortly before the planned completion of the project. In the past, it was not easy to test the system before everything was put together. Testing folk call this the “big bang”. With the “big bang” approach, a testing effort initially planned to last a couple of weeks easily extended to 10 months. It is expensive, unpredictable – and still dangerously popular.

There is a good reason for testing early: problems are cheap and fast to fix when identified early on. The root cause may be somewhere deep in the system design. Finding it, correcting it, and verifying the correction may take a lot of effort and may consume even more time. What’s worse, such rectifications are likely to introduce a bunch of new problems.

Assume one programmer made an error and five other programmers wrote new code based on the erroneous code – and assume the new code was correct. Once that single error is found and corrected those five other pieces of code will probably each behave erroneously. Imagine this happening a few hundred times. The result is a huge and often panicky test-debug-correct-retest circus that introduces new errors while correcting old ones. This is how the “big bang” works.

The obvious solution is to shift testing left, i.e. test at much earlier stages of system development and much closer to the code. Defects will then be detected when they have not yet affected the rest of the system. Obviously, they are then faster to fix and verify. When it’s possible to spread the testing effort over a longer period of time, the total number of people needed for the process will be smaller.

Agile development methods, modern tools, and cheap computing capacity have eliminated all the old excuses for not testing early. If you want to know how professional your agile teams are, find out how they test…or if they test.

But the evolution did not stop there. The internet, the cloud, and agile methods have facilitated a whole new way of architecting and releasing software. Information systems are no longer independent, slowly-changing monoliths. They are being put together from seemingly independent elements and released frequently.

For example, your ERP is likely to be linked to your CRM. The ERP may be inside your network but the CRM may reside in the cloud. The ERP probably serves several front-end applications developed independently, and it frequently exchanges information with your suppliers’ and resellers’ systems. Whenever one of these systems changes, your critical business processes may be affected.

Somewhat surprisingly, there is also a need to shift testing right, too. Right-shifting means, in practice, executing tests in the production environment. Often, the fastest and cheapest way to ensure everything is still working is to keep running tests in production and to monitor how the systems behave in relation to the business processes they are meant to serve.

Shifting testing left improves the speed, quality, and productivity of software development. Shifting testing right improves confidence in business process performance among different integrated information systems.

Check out the rest of the blog series here:

  1. Focus on risks and dependencies
  2. Train your digital operations people in DevOps
  3. Start viewing your IT investments as products rather than projects
  4. Shift testing left – and shift testing right
  5. Automate
  6. Measure
Esko Hannula

Written by Esko Hannula