Container Node File Structure¶

Automation provides several analytic container images, pre-configured and ready to support any database, computing, programming, provisioning, data visualization, reporting, and business intelligence tools an organization uses.

The images are built with the DataKitchen Interface Layer and require specific files and file structures to be present in a container node. Note that custom images may not have the same requirements.

Note

The following files and folder descriptions are necessary for all container nodes. More details on the files and folder structure specific to the GPC can be found in the General Purpose Container documentation.

Container node structure¶

node_name

Warning

If the container node file structure is not maintained, the behavior of the node is not guaranteed.

description.json¶

Container nodes require a node-level description.json file. This system file sets the node type, provides a description field to describe a node's purpose, and sets an icon for the graph view of the container node (if this is not the default Docker icon).

{
    "type": "DKNode_Container",
    "description": "[YOUR DESCRIPTION HERE]",
    "icon": ""
}

notebook.json¶

All container nodes (using either Automation analytic container images or custom images) require a node-level notebook.json file. This system file includes the configuration properties of the container itself.

See Container Notebook Properties for field descriptions and requirements.

{
    "image-repo"                 : "ac_process_info_container",
    "image-tag"                  : "latest",
    "dockerhub-namespace"        : "datakitchen",
    "dockerhub-username"         : "{{dockerhub.username}}",
    "dockerhub-password"         : "{{dockerhub.password}}",
    "analytic-container"         : true,
    "container-input-file-keys"  : [
      {
        "key"       : "inputfiles.some-input",
          "filename"  : "some_input_file.csv"
      }
    ],
    "container-output-file-keys" : [
      {
        "key"       : "outputfiles.some-output",
        "filename"  : "results.xml"
      }
    ],
    "assign-variables": [
      {
        "name"      : "rowcount",
        "file"      : "rowcount.txt"
      },
      {
        "name"      : "successcount",
        "file"      : "successcount.txt"
      },
      {
        "name"      : "failurecount",
        "file"      : "failurecount.txt"
      }
    ],
    "inside-container-file-mount"            : "/dk/ContainerWorkingDirectory",
    "container-output-log-file"              : "ac_logger.log",
    "delete-container-when-complete"         : false
}

Every container node contains a docker-share directory that stores the scripts the node runs.

See GPC docker-share Directory for information specific to the GPC container node.

docker-share/config.json file¶

Every container node that uses a DataKitchen container image must have a config.json file in its docker-share directory and must use the structure shown below.

Note that custom container images may not require this file.

{
    "apt-dependencies": [ ],
    "dependencies": [ ],
    "keys": {
        "run_script": {
            "script": "",
            "environment": {},
            "parameters": {},
            "export": [ ]
        }
    }
}

Custom files¶

You can create new container files or upload external files to use in a container node. The system stores all custom files in the /docker-share directory in the container. All file types are accepted.

To add files to a container node, open the Node Editor, select the Configuration tab, and click Add Container File.