GPC notebook.json

The required notebook.json file defines the container configuration.

notebook.json properties and descriptions

Property	Description and comments
image-repo	A string defining the Docker Hub image repository to use. The GPC value is dk_general_purpose_container.
dockerhub-namespace	A string defining the Docker Hub namespace to use. The GPC value is datakitchenprod.
image-tag	A string defining the Docker Hub image tag to use. The GPC default value is latest. Best practice is to use a fixed version.
dockerhub-username	A string defining the Docker Hub username, generally set in kitchen vault secrets.
dockerhub-password	A string defining the Docker Hub password, generally set in kitchen vault secrets.
analytic-container	A boolean that defaults to true for containers that incorporate the DataKitchen Interface Layer.
tests	A dictionary of tests to apply to container variables. This parameter is optional but strongly recommended. DataKitchen recommends building tests on all graph nodes to ensure data integrity.

Visual example in Automation

The Connection tab of a container node shows the GPC settings. A public GPC image does not require credentials entry.

Default notebook.json file format

{
    "image-repo": ""
}

notebook.json example

Notes:

• container-input-file-keys provides a method of importing files into the container from data sources, and container-output-file-keys exports files containing transformed data to a data sink. This combination moves files, rather than just exporting parameter values as defined in the export fields of config.json.
• delete-container-when-complete, when set to true, instructs the process to delete the runtime container post-execution for disk space cleanup.

{
    "image-repo": "{{dockerhubConfig.image_repo.general_purpose}}",
    "image-tag": "{{dockerhubConfig.image_tag.general_purpose}}",
    "dockerhub-namespace": "{{dockerhubConfig.namespace.general_purpose}}",
    "dockerhub-username": "{{dockerhubConfig.username}}",
    "dockerhub-password": "{{dockerhubConfig.password}}",
    "container-input-file-keys": [
        {
            "filename": "{{global_superstore_orders_filename}}",
            "key": "s3_datasource.mapping1"
        }
    ],
    "container-output-file-keys": [
        {
            "filename": "{{global_superstore_orders_filename}}",
            "key": "s3_datasink.mapping1"
        }
    ],
    "delete-container-when-complete": false
    "tests": {
        "log_dockerhub_tool_instance": {
            "test-variable": "dockerhubConfig",
            "action": "log",
            "type": "test-contents-as-string",
            "test-logic": "dockerhubConfig",
            "keep-history": true,
            "description": "Logs the DockerHub tool instance."
        },
        "test_success": {
            "test-variable": "success",
            "action": "stop-on-error",
            "type": "test-contents-as-boolean",
            "test-logic": "success",
            "keep-history": true,
            "description": "Stops the OrderRun if success is False."
        }
    }
}