Ecosystem testing made easier with test automation

Sep 07, 2017 / by Kalle Huttunen

Ecosystem testing presents IT professionals with unique challenges: in an increasingly interconnected world, ecosystems are more complex and interdependent than ever. Moreover, there are no standard solutions for testing complicated systems that may feature hardware, software and multiple interfaces. 

The word ecosystem has crossed over into the tech sector where it has become a trendy, although sometimes confusing, buzzword. Popular because it describes the increasingly interdependent environment in which we operate, and confusing because it has different meanings for different groups: tech types tend and business bulls tend to have different ideas about the concept.

Nevertheless, we are all familiar with well-known ecosystems such as tech giant Apple’s, which is closed and proprietary. You may participate if you follow the rules Apple prescribes. Other ecosystems are more open – they provide some specifications, but you can enter once you comply with them, and control is looser.

Ecosystems also vary along with protocols for testing them. Many testing professionals are wary of the term ecosystem, because it’s difficult to test systems with many dependencies. In fact, we generally try to avoid them in testing.

That’s because the more systems interact to form ecosystems, the more difficult it can be to rely on test results. Software is easily modified, adding to the testing nightmare. You might test part of a system in Finland only to find that someone in the US has changed some element of it overnight.

This kind of scenario is more common today because we have fewer systems that are completely isolated. The only examples of entirely independent, stand-alone systems might be nuclear facilities and other high-security, high-risk installations.

Service virtualization a simplified approach

Although ecosystem testing may seem like a nightmare, we still need to find ways to solve the challenge. That’s because some ecosystems may be life-critical. Others may underpin essential networks such as payment systems, and testing becomes mandatory.

In such cases, we need to find ways to address the difficulties that present themselves. There is no standard solution to solve all ecosystem testing problems – and none may ever be invented, but there are ways to grapple with the dilemma.

One approach to ecosystem testing is known as service virtualization. It involves reducing dependencies and isolating one part of the ecosystem while simulating the entire entity. The simulated ecosystem would therefore be stable with predictable behavior, allowing professionals to test the isolated segment against the simulation.

However, this method is not entirely reliable because the real ecosystem may vary from the simulation. It is also a highly-simplified scenario – in reality, an ecosystem may involve hundreds of software applications, along with hardware and multiple interfaces.

Test automation to the rescue

Another method that Qentinel has been using is test automation. In circumstances where there are many dependencies and fluctuating elements, continuous testing becomes vital. In such cases, there is not enough time for manual testing, so test automation can be used to run tests repeatedly with every change that is made. Even if no changes occur in your own systems, tests help identify defects caused by variations in other software over which you may have no control.

Qentinel has provided a test automation solution where the customer’s clients -- hundreds of them -- were able to use test automation as a self-service module to execute specific tests that cleared them for access to the customer’s ecosystem as safe partners for exchanging data.

Such cases are increasing to the point where test automation standards may be developed for certain industries. One good example of this development is Apotti, a centralized data system forming the backbone of social and healthcare in the Helsinki metropolitan area.

Competitors may even agree to use a certain test automation standard to prescribe compliance for that sector. In such cases, we are likely to see a test automation solution implemented around that standard to allow for automatic software tests for players in the industry.

Kalle Huttunen is a director at Qentinel with a special focus on Qentinel Pace. He is a testing industry veteran with more than 20 years’ experience working with startups and established corporations in the field. Kalle’s training and expertise have been tried and tested in a variety of roles, including test automation architect, test strategy consultant and test manager.


Topics: Ecosystem, Test Automation

Kalle Huttunen

Written by Kalle Huttunen