File Evaluation Sequence

The following sequence describes the underlying actions performed by the system during the execution of recipe variation files.

1. Load variables.json.

   This file contains the base variable set for the recipe.

2. Process and resolve variables, merge with overrides.

   See Variable Set Hierarchy for more information.

3. Load recipe-level description.json.

4. Load graph.
5. Send run startup notification, if configured.
6. For each node in the graph, perform the following actions:
   • Process node and node steps.
   • Load sources and files.
   • Add outputs to the recipe-node dictionary, node-level description.json, and notebook.json files.
7. If the node has data sources, the following occurs for each data source in a node (indeterminate order):
   • Process and load user-named config file.
   • Validate and execute all its keys (substeps) in their defined order.
   • Run all defined tests in their defined order.
8. Execute the node's notebook.json.
   • For container nodes, process files in /docker-share and put them into the container.

9. If the node has data sinks, the following occurs for each data sink in a node (indeterminate order):

   Note: If a container node fails during processing, the data sinks will not execute.

   • Process and load user-named config file.
   • Validate and execute all its keys (substeps) in their defined order.
   • Run all defined tests in their defined order.
   • Send run success or failure notification, if configured.