3.4.3.0 KPI test

KPI Test allows to measure data regarding a chosen calculation from related Data Source by using aggregation functions. 

Step 1:

Choose/Create a Data Set for the Test.

Step 2:

Select the KPI test type, then click Edit using the pencil icon.

There are two options to perform a KPI Test:

2.1. KPI Range - alert when KPI values are out of an expected number or percent value range.

for each column that participates in the Test and is not a key, a suitable aggregation function must be selected (e.g., SUM, AVG, MIN, or MAX). If more than one key is chosen, rows will be grouped based on the selected keys, and the aggregation function will be applied accordingly to compute the desired KPI values.

2.2. KPI Snapshot - alert when KPI values differ from the previous run snapshot(s). you can choose between the Last snapshot & Multiple snapshots:

Number of snapshots - in case "Multiple snapshots" option is chosen, the number of snapshots to compare to, needs to be selected. 

Calculation: 

There are 4 options available in Calculation: Avg, Min, Max, Median.  Chosen calculation option(s) will be applied according to the number of snapshots.

Ignore executions with alert -  executions with an alert/warning will be skipped for calculation.

Ignore executions with failure - executions with failure will be skipped for calculation.

Step 3:
User can define Global Rule that will affect all data or specific rules for specific rows.

When set up is finished, you can run the Test by clicking "Run Test".

When test type is KP Snapshot => in the results you'll see the comparison between the snapshot results: