Security Considerations

To maintain adequate security, certain behaviors exist when using secrets.

1. No vault secrets in SQL scripts

In order to maintain proper security, vault secrets in SQL scripts will not work.

Do not add a direct vault secret or a parameter defined as a vault secret into scripts. This restriction applies to SQL resource files as well as to inline key queries for database sources and sinks in JSON files.

If the system encounters a vault secret inside SQL, it ignores it and never resolves the value to retrieve the toolchain credentials stored in the vault. While users may find a native SQL error in logs when this happens, the system does not generate a user-facing error message.

Order runs fail with secrets in SQL. As a security measure, the system does not allow any reference in a SQL script or inline SQL query that will resolve to a vault secret. The system ignores these entries, and the order run fails without the proper values.

2. Vault secrets in container nodes

When building container nodes in recipe variation graphs, secrets must be passed to the container as parameters in config.json. This method is the most secure and only approved way of injecting secrets into container processing.

Sample config.json

{
    "apt-dependencies": [ ],
    "dependencies": [ ],
    "keys": {
        "tableau_script": {
            "script": "tableau_test.py",
            "environment": {},
            "parameters": {
                "PASSWORD": "#{vault://tableau/password}"
            },
            "export": [
                "success"
            ]
        }
    }
}

3. Container failure behavior

Secrets cannot be inserted in container scripts directly by referencing a vault path or variables.

Any supported container that uses the standard DataKitchen Interface Layer will not resolve secrets outside of the parameters field in the container configuration. An order run will fail on a container node that has these references.

• Container nodes using images that include the DataKitchen Interface Layer (for example, the General Purpose Container image) generate warning messages in order run logs to help users find the problems.
• Container nodes using container images that do not include the DataKitchen Interface Layer will fail with tool-specific errors rather than DataKitchen platform errors.

Order runs will fail with secrets in container scripts. The system does not allow secret resolution within container scripts. Secrets must be passed using vault paths in config.json parameters.

You can identify a container node that includes the DataKitchen Interface Layer because it has the "analytic-container": "True" entry in its notebook.json configuration.