Declare Tests

Tests can be declared:

• In specific data sources and data sinks.
• In a recipe node itself.

Tip

Users have no restrictions when it comes to naming tests in the UI. Because a goal of DataOps is to use automated tests to provide high coverage of recipes, clearly named tests help rapid debugging in the event that a test fails. As best practice, test names should adhere to Automation's naming conventions.

Tests in sources and sinks

Tests declared in a node's data sources or sinks are recorded in the applicable source.json or sink.json file. These tests evaluate built-in runtime variables generated in the source or sink.

How to

1. Use one of the following to navigate to the Tests tab of the Node Editor:
   • From the Recipes page, select the Nodes tab, then select a node.
   • From a variation's graph, select a node. Click Edit Node.

2. To create a new test:

   Click Add Source Test or Add Sink Test and enter your test details.

3. To edit an existing test:

   Search for and select the test, then update the test details.

4. Click Copy next to a test to duplicate its configuration within the node.

   Adjust the configuration to make the new test or step unique.

5. Click Delete to remove a test from the node.

6. When done, select Add notes for changelog , describe the changes, then click Update.

For help filling in the test details, follow the descriptions in Test Properties.

Visual example in Automation

The following visual examples show how tests in a DataMapper node use runtime variables to ensure that data pulled from an S3 source meet the expected results.

A test created for a runtime variable value storing a file count from both wildcard and single-file mappings from an S3 data source:

A test created for a file size runtime variable from a single-file mapping from an S3 data source:

JSON example

The following JSON example shows how tests in a DataMapper node use runtime variables to ensure that data pulled from an S3 source meet the expected results.

{
    "name": "source",
    "type": "DKDataSource_S3",
    "config-ref": "s3Config",
    "wildcard": "*",
    "wildcard-key-prefix": "{{s3Config.raw_path}}",
    "set-runtime-vars": {
        "key_count": "count_s3_files"
    },
    "keys": {
        "mapping1_source": {
            "file-key": "s3_source/path/vendor_data.csv",
            "use-only-file-key": true,
            "set-runtime-vars": {
                "size": "vendor_data_size"
            }
        }
    },
    "tests": {
        "test_count_files_pulled": {
            "action": "stop-on-error",
            "test-variable": "count_s3_files",
            "type": "test-contents-as-integer",
            "test-logic": {
                "test-compare": "greater-than",
                "test-metric": 0
            }
        },
        "test_vendor_file_greater_than_75mb": {
            "action": "stop-on-error",
            "test-variable": "vendor_data_size",
            "type": "test-contents-as-integer",
            "test-logic": {
                "test-compare": "greater-than",
                "test-metric": "78,643,200"
            }
        }
    }
}

Tests in recipe nodes

Tests declared generally in a node are recorded in the node's notebook.json file. These tests evaluate the results of script execution or runtime variables generated in the node.

How to

1. Navigate to the Tests tab of the Node Editor.
   • From the Recipe page, select the Nodes tab, then select a node.
   • From a variation's graph, select a node. Click Edit Node.

2. To create a new test:

   Click Add Test and enter your test details.

3. To edit an existing test:

   Search for and select the test, then update the test details.

4. Click Copy next to a test to duplicate its configuration within the node.

   Adjust the configuration to make the new test or step unique.

5. Click Delete to remove a test from the node.

6. When done, select Add notes for changelog , describe the changes, then click Update.

For help filling in the test details, follow the descriptions in Test Properties.

Visual example in Automation

The container's config.json calls the script and exports the custom variable:

A test created for a value exported by script execusion inside a container node:

JSON example

Tests declared generally in a node are recorded in the node's notebook.json file.

{
    "image-repo": "{{dockerhubConfig.image_repo.general_purpose}}",
    "image-tag": "{{dockerhubConfig.image_tag.general_purpose}}",
    "dockerhub-namespace": "{{dockerhubConfig.namespace.general_purpose}}",
    "container-input-file-keys": [
        {
            "filename": "SARIMA_ML_Model.ipynb",
            "key": "dict_datasource.resources_basic"
        },
        {
            "filename": "{{db_sales_filename}}",
            "key": "redshift_datasource.Load_Sales_Data"
        }
    ],
    "container-output-file-keys": [
        {
            "filename": "SARIMA_{{quarter}}_*.png",
            "key": "s3_datasink.*"
        },
        {
            "filename": "{{db_sales_filename}}",
            "key": "s3_datasink.Upload_Db_Sales_File"
        }
    ],
    "tests": {
        "validate_predicted_total_sales_rmse": {
            "description": "RMSE for total data",
            "action": "stop-on-error",
            "test-variable": "predicted_total_sales_rmse",
            "type": "test-contents-as-float",
            "test-logic": {
                "test-compare": "less-than",
                "test-metric": 4000
            }
        }
    }
}