7 December 2020 Release Notes

Versions 1.1.192.2, 1.1.193. NOTE: This release was delayed from its original 23 November release date.

Feature Highlights

Stay tuned for new DataKitchen features in coming releases!

Other Fixes & Enhancements

Write Method Updated for SFTP Sinks: Users reported that they received "permission denied" errors from order runs attempting to write output files to an SFTP server. The access problem occurred because the system was writing to a file on the server twice; the second attempt became an "overwrite" operation and was denied for certain FTP servers. This issue was resolved immediately with a refactor of the write method for SFTP sinks. [DEV-7351]

Resource Files Named in Compile Errors: Users found that validation errors during order submission incorrectly named node files, such as notebook.json, instead of resource files where the errors actually existed. The problem occurred when the system attempted to compile one file that had a Jinja template referencing a second file. This issue has been resolved with updated messaging and file references. Users will now see validation errors such as load_text: Invalid jinja template test.txt: <error detail>. [DEV-5861]

Downtime from Failed Orders Resolved: When scheduled orders for a single customer account failed many times in succession, no other orders could be run for that account. This problem occurred when the scheduled jobs failed while creating the runtime containers necessary for processing and used up all available memory. This issue has been resolved by a method to automatically clean up the unused containers. [DEV-7074]

Platform Handles Quotes in CSV Source: Order run logs recorded syntax errors from order runs that copied CSV files containing quotation marks (double quotes) from data sources to database tables in data sinks. This issue has been resolved with an update for escaping the quotation mark character. [DEV-7271]

Oracle Sources/Sinks Work Properly with SQL*Plus: Users experienced instances where and Oracle data source or sink would hang indefinitely because of a potential connection issue. The problem occurred with the use of the SQL*Plus command line utility that, in these cases, would remain in a waiting state for some input. This issue has been resolved with a code update to close the utility gracefully. [DEV-7172]

Nodes Using Snowflake Report Invalid SQL: Nodes using Snowflake Data Warehouses as sources or sinks were not failing on invalid SQL syntax. Users found that action, container, and DataMapper nodes were running successfully despite trace level entries in the logs that indicated SQL errors. This issue has been resolved with an update to the Snowflake connector. [DEV-7182]

Node Status Updated when Resources Exceeded: When order runs failed because they used more than the memory allocated, the Order Run Details page was showing nodes as still running. This issue has been resolved so that the status of the order run and any nodes that were running at time of failure are all displayed as failed. [DEV-6983]

Platform Renames Ingredient in Ingredient Variation Copy: When a user copied an ingredient variation, giving the copy a new variation name, the underlying ingredient listed in variations.json was retaining the original name. With two ingredients bearing the same name, the system would not allow any edits to either the original ingredient or its copy. This issue has been resolved by creating the ingredient copy with the new name of the variation copy. [DEV-7190]

Users Can Dismiss Session Timeout to Copy Cached Work: In response to user requests, this release introduces the ability to close the web app's Session Timeout dialog. This function allows users to access their work on the current platform page that may be lost when they log in again. For example, users working in the Node Editor or File Editor can dismiss the dialog and still access the tabs or files to copy any unsaved work. [DEV-7053]

Variables Editor Allows Empty Values: Variable values defined as empty strings in variables.json were generating errors in the Edit Variables dialog accessed from the Recipes page in the Variable Sets tab. Additionally, when the editor was used to create new variables, it was removing any existing variables that had empty strings. This issue has been resolved with an update to the editor allowing for empty string values. [DEV-7289]