31 October 2021 Release Notes

Version 1.1.232

Note: While 31 Oct is the expected release date, release delays may occur.

Feature Highlights

Automation Introduces Conditional Nodes!

Note

Note: This feature is released as a beta test candidate and is available only to select customers at this time.

Recipe builders now have an easy method to insert a decision point in their graphs! Conditional Nodes evaluate statements based on variable input, operators, and metrics. The results of the evaluations determine which downstream graph nodes will be executed or skipped.

This first release of the feature supports allowing or blocking the processing of one or more nodes, and condition statements that resolve to Boolean values. See Conditional Nodes for more information. [DEV-8113, DEV-7992, DEV-7830, DEV-8294, DEV-9193]

The new conditional nodes feature determines if downstream nodes are executed or skipped.

Other Fixes & Enhancements

Jinja Ignore Function Added to Container Settings: Users requested a way to prevent Jinja processing in specific container files when they were employing a downstream data build tool that transformed Jinja, instead. This release introduces the Jinja Ignore Patterns field in the Settings tab of a container node to instruct the platform to avoid compiling specified files. Users can enter multiple paths and filenames or wildcard patterns in a comma-separated or line-break list. See Container Notebook Properties for more information. [DEV-9275, DEV-9296]

New container setting flags files to skip in Jinja processing.

Snowflake Connector Refactored for Data Sources: A customer found that the DataKitchen Snowflake connector's use of the Snowflake command line interface was interfering with an HTTP proxy and failing to update SnowSQL. The result was a failure to connect to the customer's Snowflake database. This issue has been resolved for Snowflake data sources by connecting via Snowflake's Python package instead. The same refactoring will be applied to data sink connectors in a future release. [DEV-9269]

Order Runs Fail Properly on Placeholder Values: A user reported an order run hanging when a placeholder value for a required container node variable had not been updated. This and similar order runs should fail gracefully when processing a problem container node, and they should generate the appropriate errors to inform users of the problems. Instead, the system was mishandling some container node errors, creating new order runs, and updating their statuses to "Failed." This issue has been resolved, and the system fails the original order runs as expected after a standard number of retries. [DEV-8608]

Order Run Alerts Sent in System Problem Scenario: Infrequently, order runs trigger failures in the platform's Kubernetes worker nodes that run the orders. In these scenarios, the system was failing the order runs but never sending the related email notifications. This issue has been resolved, and the system sends failure alerts when order runs fail as a result of system problems as well as order processing issues. [DEV-7988]