Skip to content

Runtime Resources

Recipes contain at least one configured resource allocation that assigns memory and disk space on the agent for each order run that's processed. These upper bounds are configured to prevent runaway resource consumption.

If multiple orders run at the same time, an order that has insufficient resources stays in a pending state until resources become available in the agent. If the agent runs out of memory or disk space completely, the order will end with errors.

Memory

The default memory runtime setting allocates 1024 MB of memory for each order run. This is sufficient memory for most order runs.

This order run setting is configured in the variations.json file as the max-ram value or in a schedule form with the Max RAM field.

The memory required for an order run is equal to RAM used for processing the order + active memory caches + some Docker overhead.

Disk space

The default disk space runtime setting allocates 2048 MB of disk space for each order run. This is sufficient disk space for most order runs.

This order run setting is configured in variations.json file as the max-disk value or in a schedule form with the Max Disk Space field.

The disk space required for an order run is equal to space needed for any write operations + space for the recipe file sizes.

The disk space usage reported in logs does not include any resource requirements of shared infrastructure, such as container images and shared volumes used for file exchange within a node.

Settings for order runs and container nodes

For resource-intensive order runs—for example, those with large queries, heavy use of variables, or data sources and sinks—you can assign memory and disk space. These allocations can be set for the order run or for container nodes.

  • Resources for order runs: these variation-level resource settings affect an overall order run and the temporary storage used by data sources and data sinks; they are not relevant to processing that occurs inside a container node. Runtime settings for order run resources are configured in the UI schedule forms or in the variations.json file.
  • Resources for container nodes: these container settings only affect a container node itself. Here, boosting RAM and disk can help with heavy processing needs by running scripts within that container. Assign resource values from the Configuration tab of the Node Editor.

Warning

Editing resource allocations for active orders: edits to resource allocations have no impact on existing, active orders. When editing the resource allocations for a recipe with active orders, the existing orders must be stopped and new orders submitted after the settings have been applied.

Best practices

The majority of orders are well-handled by default resource allocation and don't need additional resource consideration. In cases where resources are constrained, follow this workflow to determine the correct configuration:

  1. Review order run logs to find clear error messages regarding resource constraints.
  2. Note the cumulative memory and disk space consumed at the point of order run failure.
  3. Check the order run graph failure point for components with memory leaks and/or excessive abnormal volume of data collection.
  4. Increase the resource allocation in the recipe variation's schedule settings.
  5. Cancel any scheduled orders for the recipe variation.
  6. Run a test order. If passing, reschedule the main order. If failing, repeat the steps above.

Tip

Use an empty schedule to set run once resources. When you apply a schedule to a variation and activate that scheduled order, the system will allocate resources—up to the maximum 1024 MB of RAM and 2048 MB of disk space—to every order run done for that schedule. To set resource maximums for run-once orders, create a schedule without a frequency and only enter values for RAM and disk space.

Container nodes

On your local machine, examine the memory consumption of any scripts you want to insert into container nodes, in a recipe. Memory leaks affecting these standalone scripts will also impact any order runs that include these scripts.

Ingredient nodes

The use of ingredients can require resource allocation consideration. When an order run encounters an ingredient node, a temporary kitchen is generated and the ingredient (referenced by the ingredient node) runs there. Its status, runtime variables, and logs are communicated back to its parent order run upon completion. As a result, the use of ingredients requires allocating enough resources to cover the distinct order runs associated with ingredient nodes.

Troubleshooting: Insufficient Resources