Test case deviation detecting and visualizing PaceEditor 

May 24, 2021 / by Kimmo Nupponen

As described in the earlier Qentinel blog postPaceEditor of Qentinel Pace is the next-generation highly advanced test development environment for PaceWord and Robot Framework test scripts leveraging a massive amount of test case data when it’s guiding the user through the scripting process. 

As alluded to in earlier blog post, PaceEditor has several advanced capabilities that help and guide you through the test development and test scripting process. One of those core functions is the capability to predict your next actions when you are defining the test scripts, which are extremely valuable and useful in guiding the user to develop test scripts further. However, predictions as such are not enough as you can trivially decide to define a test behaviour not directly predicted by the editing environment, and the question becomes how can you quickly and easily make sure that those updates are consistent?

To answer this question, PaceEditor provides support for detecting any behavioural deviations in the observed logic in the test scripts from what was expected from the “normative models” of test cases. The results of the analysis and detection process are visualized in a form of a heat map in the PaceEditor user interface in real-time as you continue to work with the script. 

Intuitively you can think of this in a way that predictions that the editor creates guide you to a certain beneficial or “normative” direction while the deviations are used to guide you away from a potentially inconsistent behaviour; away from thin ice, so to speak. 

Conventional editors can detect syntactical and semantical errors in the script and in real-time notify the user of these detected errors. However, this deviation detection capability is in stark contrast with these conventional editor features as the editor can identify behavioural deviations from a separately specified norm and then let you, the user, know about these deviations. This allows early detection of anomalous scripting and detects logical inconsistencies that are not detected by conventional editors. Technically PaceEditor implements the deviation detection capability using a collection of advanced machine learning algorithms all trained with a massive set of test case data. 

The results of the deviation analysis are visualized as heat maps in real-time in the editor user interface where the “heat” is defined as the distance between the observed and expected assets in the script in such a manner that larger distance will be emphasized over shorter distances in the editor user interface. This allows you to focus on the most important aspects of the scripts and to quickly grasp the “hot spots” in the scripts that may require immediate attention. 

Already this one particular “feature” offers a significant improvement in the user’s operation by providing you a level of understanding about the logical script content. This not only improves the user’s productivity but also has the potential to increase the quality of the scripts created by you. The quality improvement is a direct consequence of the capability to detect unexpected logical behaviour in the scripts. 

PaceEditor is part of Qentinel Pace and you can directly access it using your favourite browser by simply logging in to Pace and navigating to your test suite. If you have no account to Pace already today, you can quickly create one and start using Pace for free. You can also install PaceEditor directly to your VSCode editor from Marketplace and use it for local test development. Whatever you prefer for test development!

Start Testing for Free

Check out the rest of the blog series here:

  1. PaceEditor available as a VSCode extension
  2. Test development in Qentinel Pace
  3. Predict the most likely test steps in your test cases with ML
  4. Test case deviation detecting and visualizing PaceEditor 

Topics: Software Testing, Test Automation, Software Development, Test Creation

Kimmo Nupponen

Written by Kimmo Nupponen