Skip to content

Order Run Execution

The following provides a summary of what happens during an order run.

Terms

Note that there are several references to containers:

  • The RecipeRunner container is the Kubernetes pod that runs the order and supplies runtime resources.
  • A container node is a type of node type available to use in recipe graphs. It uses a Docker container at runtime to package code, runtime scripts, tools, and other settings.
  • Kubernetes executes container nodes in runtime containers (i.e. pods) separate from the rest of an order run, allocating separate resources to process them.

For more information, see Automation Architecture.

Order run execution diagram

An order run proceeds in more-or-less in the following order, with some actions occurring at the same time as others.

(Diagram of order run process)

  1. A user starts a variation run or submits an order.
  2. The system validates the recipe configuration, JSON, and secrets.
  3. The system fetches kitchen data from MongoDB and recipe files from Git.
  4. The system stores order information in MongoDB.
  5. The system creates a job in the Kubernetes cluster. Resources are located.
  6. The RecipeRunner container starts. Variables merge.
  7. The RecipeRunner creates order run logs in MongoDB.
  8. The RecipeRunner fetches recipe files from Git.
  9. The RecipeRunner writes order run status and data to MongoDB throughout the order run process.
  10. Order run alerts are sent throughout the process.
  11. The Recipe Runner runs nodes and compiles JSON files.
    • Sync nodes: run tests in sequence.
    • Action nodes: run sources in sequence; connect to source tools; retrieve files and run query; write results and data; run tests in sequence.
    • DataMapper nodes: run sources in sequence; connect to source tools; retrieve files and run query; generate mappings; store mapped files; run sinks in sequence; connect to sink tools; put files or data; run tests in sequence.
    • Container nodes: run sources in sequence; connect to source tools; retrieve files and run query; run container notebook; resolve secrets; compile docker-share files; store input files; store output files; stop the container as needed; run sinks in sequence; connect to sink tools; put files or data; run tests in sequence.
    • Ingredient nodes: run the ingredient notebook; get ingredient settings; create temporary kitchen; apply inflowing variables; run ingredient order; delete temporary kitchen; embed logs and test data; set outflowing variables; run tests in sequence.
  12. As needed, secrets are retrieved from Vault.

  13. For container nodes: separate resources are found to run the container. The image is pulled and the container is started. The docker-share directory is mounted and the command run in the container.

  14. When done, graph execution stops.

  15. Results display in the UI.

A note on schedules

Note that if an order run misses its scheduled start time for any reason, the system will continue to attempt to execute the order for up to 120 seconds.

File Evaluation Sequence

Runtime Resources

Recipe Nodes

Graph Processing