Built-In Runtime Variables

Data sources and sinks provide a number of optional built-in runtime variables as a result of their execution, which can be used in automated tests for quality assurance.

In JSON node files, these variables can be used in configuring the source or sink, itself, and for testing wildcard mappings.

Built-in runtime variables are explicitly declared in data sources and data sinks via the set-runtime-vars property.

Note

See File-Based Variables and Database Variables for variables used in configuring the keys (steps) within a source or sink mapping definition.

Access runtime variables

Runtime variables can be found in the Mappings tab of the Node Editor and in the JSON file of the File Editor.

Runtime variables for file-based sources and sinks

For DataMapper nodes and container nodes, runtime variables refer to "keys" used in mappings, while for action nodes the variables refer to "steps."

Runtime variables field in the UI	Runtime variables field in JSON	Description and details
N/A	set-runtime-vars	The system generates the field when a runtime variable form is completed. Used to declare a runtime variables set equal to built-in variables. Can be applied at the source/sink level or the key level. See File-Based Variables for more information. Value type: dictionary Node file: Recorded in /data_sources/source.json and /data_sinks/sink.json
Key/Step Count	key_count	Exports the count of JSON keys declared in a data source or data sink configuration. Each mapping (DataMapper or container node) or step (action node) counts as one key. Wildcard mappings dynamically generate one key per file that matches the wildcard. Single-file mappings and steps produce one key per mapping/step, found under the keys object in source and sink JSON files. This variable counts any dynamic "wildcard keys" plus any mapping keys to obtain its value. Value type: string Node file: Recorded in /data_sources/source.json and /data_sinks/sink.json
List of Key/Step Names	key_names	The JSON key names associated with node mappings or steps in a source or sink. This variable stores the file paths for both wildcard and single-file mapping keys. Value type: string Node file: Recorded in /data_sources/source.json and /data_sinks/sink.json
List of File Paths in Keys/Steps	key_files	The file paths found in JSON keys for specific node mappings/steps or that match a wildcard. Availability: File-based data sources and sinks only; not supported for Dictionary sources and sinks. Value type: string Node file: Recorded in /data_sources/source.json and /data_sinks/sink.json
Keys/Steps JSON	key_map	The JSON dictionary content from the keys object within source.json or sink.json. This variable stores each entire JSON object from every wildcard match and single-file mapping or step. If a data source or sink uses wildcards, it will be populated with the keys generated after wildcard is processed. Value type: string Node file: Recorded in /data_sources/source.json and /data_sinks/sink.json

Runtime variables for database sources and sinks

For DataMapper nodes and container nodes, runtime variables refer to "keys" used in mappings, while for action nodes the variables refer to "steps."

Runtime variables field in the UI	JRuntime variables field in JSON	Description and details
N/A	set-runtime-vars	Web app generates the field when a runtime variable form is completed. Used to declare a runtime variables set equal to built-in variables. Can be applied at the source/sink level or the key level. See Database Variables for more information. Value type: dictionary Node file: Recorded in /data_sources/source.json and /data_sinks/sink.json
Key/Step Count	key_count	Exports the count of JSON keys declared in a data source or data sink configuration. Each mapping (DataMapper or container node) or step (action node) counts as one key. Value type: string Node file: Recorded in /data_sources/source.json and /data_sinks/sink.json
List of Key/Step Names	key_names	The JSON key names associated with node mappings or steps in a source or sink. Value type: string Node file: Recorded in /data_sources/source.json and /data_sinks/sink.json
Keys/Step JSON	key_map	The JSON dictionary content from the keys object within source.json or sink.json. Value type: string Node file: Recorded in /data_sources/source.json and /data_sinks/sink.json

Runtime variables example

The example below is for a file-based FTP data source. Here, built-in variables are declared both at the source-level for files picked via wildcard and at the key-level for the file picked via an explicit key.

ftp_datasource.json

{
    "type": "DKDataSource_FTP",
    "name": "ftp_datasource",
    "config-ref": "ftpconfig",
    "set-runtime-vars": {
        "key_count": "count_ftp_files"
    },
    "wildcard": "*",
    "keys": {
        "example-explicit-key": {
            "file-key": "example.csv",
            "use-only-file-key": true,
            "set-runtime-vars": {
                "row_count": "ftp_example_row_count"
            }
        }
    },
    "tests": {
        "test-count-files-pulled": {
            "test-variable": "count_ftp_files",
            "action": "stop-on-error",
            "test-logic": "count_ftp_files = 1 or count_ftp_files > 1"
        },
        "test-row-count":{
            "test-variable": "ftp_example_row_count",
            "action": "stop-on-error",
            "test-logic": "ftp_example_row_count > 100"
        }
    }
}