quilliup allows to define relevant statuses and ranges for Test results. Thresholds play a crucial role in determining the outcome of Tests by establishing criteria for "Passed", "Warning", or "Failed" statuses based on various rules.
Types of Thresholds
-
Test-Level threshold (A) – define how many records trigger a "Failed", "Warning" or "Passed" Test result.
For example, by default a Compare Test will fail if any differences are found between the data sets. The default threshold is set to 0 (Zero) with the option "High is Bad", meaning that even a single differing row will cause the Test to fail. -
Column-Level threshold (B) – thresholds set on the column's level work on either the values themselves or the number of differing records, depending on the type of Test.
For example, in a Compare Test that scans 100 records and finds 7 differences, the Test would fail if the threshold is set to 0 (Default). You can define a number of records or a percentage of total records as the threshold for triggering a warning or failure. For example, if the threshold is set to 10%, the Test would pass in this case.
You can configure which fields are compared, as well as set individual thresholds for each field.
By default, every field uses a bidirectional "Center Is Good" threshold set to 0. This means that a difference of 0 is required for the comparison to pass; otherwise, the row is considered unsuccessful.
The bidirectional setting allows for differences either above or below the threshold. If you want to trigger a failure only when the difference exceeds (or falls below) a certain value, you can set a one-way threshold instead. In all cases, differences are measured relative to the master dataset defined in the Compare rule.
Field-level thresholds determine whether an individual record passes or fails, while the Test-level threshold determines whether the entire Test passes or fails, based on how many records failed.
General Definitions
Percent / Number - failed records will be measured by percentages or by numbers, according to your choice.
Rule Type - One way or Two ways:
1). One Way - the Test outcome (pass or fail), depends on whether the value exceeds or falls below the defined limit.
- High is good (1):
If the Test's result exceeds the specified value, the Test will be considered a pass. This type of threshold is used when higher results indicate better performance.
Failed =< value < Passed
Example:
Failed =< 5 < Passed
Any positive value higher than 5 => the Test passes. A result of 5 or less => the Test fails.
- High is good with alert (2):
If the Test result exceeds a specified value, the Test will pass. If the result falls within a lower range, the Test will pass with a warning. This type of threshold is useful when higher results indicate better performance.
Failed <= left value < Alert <= right value < Passed
Example:
Failed <= 0 < Alert <= 5 < Passed
0 and below - the Test fails, between 0 and 5 (included) => Test ends up with a warning. A value of 5 and above => the Test passes.
- High is bad (3):
If the Test result remains below the specified value, the Test will pass. This Threshold type is relevant for cases where lower results are preferred or indicate better outcomes.
Passed <= value < Failed
Example:
Passed <= 0 < Failed
Any positive value higher than 0 => the Test passes. 0 and below => the Test fails.
- High is bad with alert (4):
If the Test result remains below a specified value, the Test will pass. However, if the result falls within a higher range, the Test will pass with a warning. This type of Threshold is useful when lower results indicate better performance.
Passed <=left < Alert <= right < Failed
Example:
Passed <= 1 < Alert <= 5 < Failed
1 and below => the Test passes, between 1 and 5 (included) => the Test ends with a warning. A value of 5 and above => the Test fails.
2). Two Ways - the Threshold is determined based on a "Two-Way" Rule, where the Test outcome - pass or fail - depends on whether the value falls within the specified required range. This approach works well for absolute values as well:
- Center is bad (5):
If the values are below the left boundary or above the right boundary ,the test will pass. However, if the values fall within the range between the left and right boundaries, the Test will fail.
Passed < left <= Failed <= right < Passed
Example:
Passed < 0 <= Failed <= 0.001 < Passed
Results between 0 and 0.001=> the Test fails, below 0 or above 0.001 => the Test passes.
- Center is bad with alert (6):
If the values are below the left boundary or above the right boundary, the Test will pass. However, if the values are close to either boundary but still within the range, the Test will pass with a warning. If the values fall entirely within the specified range, the Test will fail.
Passed < left <= Alert < left middle <= Failed <= right middle< Alert <= right < Passed
Example:
passed< -5 <= alert < 0 <= failed< =0 <= alert < 5 < passed
Test results below -5 and above 5 => the Test passes. For result 0 the Test fails, and below and above - the Test ends with an alert.
- Center is good (7):
If the values are within a specific range, either above or below the boundaries, the Test will fail or trigger an alert. This type of Threshold works well for absolute numbers:
Failed < left <= Passed <= right < Failed
Example:
Failed < 0 <= Passed <= 0.001 < Failed
Test results between 0 and 0.001 => the Test passes, and below or over - the Test fails.
- Center is good with alert (8):
If the values are within a specific range, either above or below the boundaries, the Test will trigger a warning. However, if the values fall outside the range (either above the upper boundary or below the lower boundary), the Test will fail. This type of Threshold works well for absolute numbers:
Failed < left <= Alert < left middle <= Passed <= right middle < Alert <= right < Failed
Example:
Failed < -5 <= Alert < 0 <= Passed < =0 <= Alert < 5 < Failed
Test results below -5 and above 5 => the Test fails. If the Test result is 0 => the Test passes, and below and over - is an alert.
Variables
There is an option to use Variables as Threshold values:
By clicking the {;} sign, a drop-down of available Variables will appear and you will be able to choose one.
You can also manually enter the Variable name. Please ensure it is written correctly, respecting case sensitivity, with @ at the beginning and ; at the end - e.g. @VariableName;.
The Threshold is defined differently for each Test type.
For further information about Threshold definitions in various Tests, please refer the following manual: Thresholds By Test Types
Comments
0 comments
Please sign in to leave a comment.