Test Properties

The table below provides details on the properties you can configure during test creation. When tests are set up in the UI, they are recorded in JSON files.

For information on how to create tests, see Declare Tests.

UI test creation fields	JSON fields	Description and details	Required?
Test Name		Brief title for the test, up to 256 characters. Must be unique within the node. Default value: test1.	Required
Description	description	Definition to provide additional detail about the purpose of a test.	No
Failure Action	action	Options for how the system responds when a test fails. Valid values: stop-on-error: the order run will be stopped, and subsequent nodes will not be executed. warning: the order run will continue, but a warning message will display. An alert can be set for these test warnings. log: the results will be logged whether the test passes or not.	Required

UI Test Logic fields	JSON Test Logic fields	Description and details	Required?
Compare variable against metric > Test Variable	test-variable	The name of the variable whose compiled value will be compared. Use runtime variables declared in this node's steps or source/sink mappings, defined in the node's runtime variables, or declared in the variation or kitchen.	Required
Compare variable against metric > Type Conversion	type	The type of value, for example, integer or Boolean. This selection evaluates the value of test-variable and test-metric (for all types except string) as a specific data type. If a value cannot be cast to the specified type, an error will be thrown. If omitted, the user is responsible for ensuring the correctness of data types used in test logic. A test with a String selection made for any non-string test variable will process but log a warning to update the test logic to use another type conversion or None. Valid values:integer (test-contents-as-integer), float (test-contents-as-float), string (test-contents-as-string), date (test-contents-as-date), Boolean (test-contents-as-boolean), none (test-contents-as-default)	No
Compare variable against metric > Comparison	test-logic > test-compare	The operator for running the test, for example, == or >. Valid values:== (equal-to), > (greater-than), >= (greater-than-equal-to), < (less-than), <= (less-than-equal-to) JSON parent field:The test-logic object contains values for both comparison and metric.	Required
Compare variable against metric > Metric	test-logic > test-metric	A value against which to compare the test variable value. This metric option enables test results to be plotted in Order Run Test Results graphs on the Variation Metrics page. Valid values: Must contain either a control value or a historical value entry. Control Value: a variable or specified value for comparison. Can be a literal (100, "100"), a date expression, or a variable name. Variable History: compares the test variable value with the value of the same variable (historic-metric) in the previous order run of the variation (historic-calculation > previous-value) or an average value of the variable across all previous order runs (historic-calculation > running-average). See Historical Comparisons for more information. JSON parent field: the test-logic object contains values for both comparison and metric.	Required
Specify Jinja expression	test-logic	When selected, displays fields to configure a variable and Jinja expression for the test details. This expression option for test logic does not allow the system to graph the test results on the Variation Metrics page. Test Variable: the name of the variable whose compiled value will be compared. Use runtime variables declared in this node's steps or source/sink mappings, in the node's runtime variables, or variables from sets declared in the variation or kitchen. Type Conversion: the type of value, for example, integer or string. Expression: see Custom Logic Expressions and Jinja references for more information.	Not required to declare a test. If selected, Test Variable and Expression are required.
(Not applicable)	keep-history	The UI automatically generates this field in node files when a historic comparison is selected. This value determines whether the test results generated by each order run may be stored to create a history used for benchmarking. Value type: Boolean Valid values: true, false	No