Container Notebook Properties¶
The properties below are specific to container nodes. The table includes the fields seen in the Node Editor and the corresponding JSON fields.
All data is recorded in the notebook.json file.
Connection properties¶
| Node Editor Connections tab > Docker Image section | JSON field | Description |
|---|---|---|
| Namespace | dockerhub-namespace | Docker image namespace. |
| Repository | image-repo | The name of the docker image. See Container Images for more information. Required field. |
| Image Tag | image-tag | The docker image tag denoting the image version to be pulled. Default value: latest |
| Image uses a DataKitchen base image (checkbox) | analytic-container | When true, the system expects a container image built with, or based on, the DataKitchen Interface Layer. When false, the system expects a custom image, where the node file requirements depend on how the image is built. Note that all Automation a nalytic containers also expect a docker-share/config.json file with a specific structure. Non-analytic containers may not require this file at all. See Container Node File Structure for more information. Value type: Boolean Default value: true |
| Docker Registry Credentials > Username | dockerhub-username | Docker registry service user name. Required field if using Docker Registry Credentials authentication method. |
| Docker Registry Credentials > Password | dockerhub-password | Docker registry service password. Required field if using Docker Registry Credentials authentication method. |
| Docker Registry Credentials > Registry URL | dockerhub-url | The URL of the Docker registry from where images should be picked. Default value: docker.io |
Configuration and settings¶
| Node Editor Configuration tab > Container Settings section | JSON field | Description |
|---|---|---|
| Max RAM (in MB) | max-ram | The maximum amount of memory reserved for container execution. See GPC Resource Allocation. |
| Max Disk Space (in MB) | max-storage | The maximum amount of disk space reserved for container execution. See GPC Resource Allocation. |
| Command to Run | command-line | The command to be executed by the container when it is started. This value is typically not needed for analytic containers built with the DataKitchen Interface layer. |
| Working Directory | inside-container-file-mount | The working directory inside the container. Default value: Default given by the same container |
| Log File Path | container-output-log-file | The name of the log file being generated by the container. Default value: ac_logger.log |
| Jinja Ignore Patterns | jinja-ignore-patterns | The paths and filenames or wildcard patterns for container files in which Jinja expressions will be ignored during the runtime compile. Separate multiple entries with commas or line breaks. This field uses fnmatch pattern matching for wildcards, where * matches everything and ? matches any single character. Examples: /dir1/dir2/*.csv, /dir1/*, *.txt, Entries are recorded in notebook.json in the following JSON list format: ["/dir1/dir2/*.csv", "/dir1/*", "*.txt"] |
| Delete container on completion (checkbox) | delete-container-when-complete | Determines if the runtime container is deleted immediately after its processing has completed and files have been extracted. Note that runtime variables remain available for downstream processing whether the container is deleted or not. Use this option in notebook.json to prevent resource-intensive containers from depleting disk space over time. Value type: Boolean Default value: true |
| Legacy Settings > Shared Directory | inside-container-file-directory | The name of the folder that will be used to exchange information between the container and the node. It's relative to inside-container-file-mount. When analytic-container is false, this folder must be placed in the working directory. Availability: Only when the field is already present in legacy JSON code Default value: docker-share |
| Legacy Settings > Configuration Folder | container-input-configuration-file-path | The path to the folder of the files being exchanged with the container on the side of the recipe node, relative to the recipe root directory. Availability: Only appears in the UI when the field is already present in legacy JSON code. Default value: [node-name]/docker-share |
| Legacy Settings > Configuration File | container-input-configuration-file-name | The name of the configuration file for the container. Availability: Only appears in the UI when the field is already present in legacy JSON code. Default value: config.json |
| Legacy Settings > Progress File Path | container-output-progress-file | The name of the progress json file being generated by the container. This file stores the global variables exported from a container node. Automation uploads this file and converts the global variables into runtime variables during an order run. Availability: Only appears in the UI when the field is already present in legacy JSON code. Default value: progress.json File size limit: The maximum size of the progress file is 512 KB. If the file increases beyond this limit, the order run will fail. |
Inputs or outputs¶
| Node Editor Inputs or Outputs tabs | JSON field | Description |
|---|---|---|
| both: Outputs > To Variables > Variable Name Outputs > To Variables > Container File > File Path | assign-variables | A list of associations between files inside the container and variables. The JSON field for each To Variables mapping contains the file path and filename from the UI File Path field and the name value from the UI Variable Name field. Any variables defined will be loaded with the contents of these files that should be inside the container once its execution finishes. The variables can be used in tests downstream in the workflow. Decode file as JSON: A checkbox that determines if the string value for the file should be parsed as JSON (checked) before assigning it to the variable or not (unchecked). The UI default is false/unchecked. The JSON field recorded is decode-json. |
| Inputs > Mapping > all fields | container-input-file-keys | A list of mappings between data source keys and files to be placed inside the container once created. The JSON field for each input mapping contains the filename from the UI Target File Path field and a key value aggregated as source.<mapping_name>. |
| Outputs > To Sink > Mapping > Container File Path | container-output-file-keys | A list of mappings between files inside the container and data sink keys. The JSON field for each To Sink mapping contains the filename from the UI Container File Path field and a key value aggregated as sink.<mapping_name>. These files will be retrieved from the container once it finishes its execution, and sent to the data sinks. |