Functions for Variable Transformation¶

Built-in functions are available for variable transformation and can be called using Jinja syntax.

Many of these functions behave the same as Python's built-in functions. For more information, see Python's Built-in Functions documentation.

Built-in function syntax¶

When using built-in functions, only use the Jinja "{{ }}" syntax once, outside of the function.

# Proper syntax
"add_days_func": "{{add_days(now, 1)}}"

# Improper syntax
"add_days_func": {{add_days({{now}}, 1)}}

Transform datetimes¶

Function name	Description	Raw JSON examples	Compiled examples
`add_days(date, days)`	Adds days to a given datetime object; days can be positive or negative.	`"add_days_func": "{{add_days(now, 1)}}"`	`"add_days_func": "2019-02-07 01:44:37.696468"`
`add_days(date, days)`		`"subtract_days_func": "{{add_days(now, -1)}}"`	`"subtract_days_func": "2019-02-05 01:44:37.696468"`
`add_weeks(date, weeks)`	Adds weeks to a given datetime object; weeks can be positive or negative.	`"add_weeks_func": "{{add_weeks(now, 1)}}"`	`"add_weeks_func": "2019-02-13 01:44:37.696468"`
`add_weeks(date, weeks)`		`"subtract_weeks_func": "{{add_weeks(now, -1)}}"`	`"subtract_weeks_func": "2019-01-30 01:44:37.696468"`
`add_months(date, months)`	Adds months to a given datetime object; months can be positive or negative.	`"add_months_func": "{{add_months(now, 1)}}"`	`"add_months_func": "2019-03-06 01:44:37.696468"`
`add_months(date, months)`		`"subtract_months_func": "{{add_months(now, -1)}}"`	`"subtract_months_func": "2019-01-06 01:44:37.696468"`
`add_years(date, years)`	Adds years to a given datetime object; years can be positive or negative.	`"add_years_func": "{{add_years(now, 1)}}"`	`"add_years_func": "2020-02-06 01:44:37.696468"`
`add_years(date, years)`		`"subtract_years_func": "{{add_years(now, -1)}}"`	`"subtract_years_func": "2018-02-06 01:44:37.696468"`
`date`	Python object to present the value in JSON for a given date. See Python date objects.	`"date_variable": "{{date(2020, 10, 28)}}"`	`"date_variable": "2020-10-28"`
`date_format(date, format)` (weekday)	Returns a string representation of a given datetime object based on a format string.	`"weekday_func": "{{date_format(now, '%A')}}"`	`"weekday_func": "Wednesday"`
`date_format(date, format)` (weekday)		`"weekday_short_func": "{{date_format(now, '%a')}}"`	`"weekday_short_func": "Wed"`
`date_format(date, format)` (day of)	Returns a string representation of a given datetime object based on a format string.	`"day_of_week_func": "{{date_format(now, '%w')}}"`	`"day_of_week_func": "3"`
		`"day_of_month_func": "{{date_format(now, '%d')}}"`	`"day_of_month_func": "06"`
		`"day_of_year_zero_padded_func": "{{date_format(now, '%j')}}"`	`"day_of_year_zero_padded_func": "037"`
`date_format(date, format)` (week number)	Returns a string representation of a given datetime object based on a format string.	`"week_number_sunday_func": "{{date_format(now, '%U')}}"`	`"week_number_sunday_func": "05"`
`date_format(date, format)` (week number)		`"week_number_monday_func": "{{date_format(now, '%W')}}"`	`"week_number_monday_func": "05"`
`date_format(date, format)` (month)	Returns a string representation of a given datetime object based on a format string.	`"month_func": "{{date_format(now, '%B')}}"`	`"month_func": "February"`
		`"month_short_func": "{{date_format(now, '%b')}}"`	`"month_short_func": "Feb"`
		`"month_padded_func": "{{date_format(now, '%m')}}"`	`"month_padded_func": "02"`
`date_format(date, format)` (year)	Returns a string representation of a given datetime object based on a format string.	`"year_func": "{{date_format(now, '%Y')}}"`	`"year_func": "2019"`
`date_format(date, format)` (year)		`"year_short_func": "{{date_format(now, '%y')}}"`	`"year_short_func": "19"`
`date_format(date, format)` (hour)	Returns a string representation of a given datetime object based on a format string.	`"hour_24_zero_padded_func": "{{date_format(now, '%H')}}"`	`"hour_24_zero_padded_func": "01"`
`date_format(date, format)` (hour)		`"hour_12_zero_padded_func": "{{date_format(now, '%I')}}"`	`"hour_12_zero_padded_func": "01"`
`date_format(date, format)` (am/pm)	Returns a string representation of a given datetime object based on a format string.	`"am_pm_func": "{{date_format(now, '%p')}}"`	`"am_pm_func": "AM"`
`date_format(date, format)` (time)	Returns a string representation of a given datetime object based on a format string.	`"minute_zero_padded_func": "{{date_format(now, '%M')}}"`	`"minute_zero_padded_func": "44"`
		`"second_zero_padded_func": "{{date_format(now, '%S')}}"`	`"second_zero_padded_func": "37"`
		`"microsecond_zero_padded_func": "{{date_format(now, '%f')}}"`	`"microsecond_zero_padded_func": "696468"`
`date_format(date, format)` (locale datetime)	Returns a string representation of a given datetime object based on a format string.	`"locale_datetime_func": "{{date_format(now, '%c')}}"`	`"locale_datetime_func": "Wed Feb 6 01:44:37 2019"`
		`"locale_date_func": "{{date_format(now, '%x')}}"`	`"locale_date_func": "02/06/19"`
		`"locale_time_func": "{{date_format(now, '%X')}}"`	`"locale_time_func": "01:44:37"`
`date_format(date, format)` (percentage)	Returns a string representation of a given datetime object based on a format string.	`"literal_percentage_example": "{{date_format(now, '%% %Y-%m-%d %%')}}"`	`"literal_percentage_example": "% 2019-02-06 %"`
`date_parse(date_string, format)`	Parses a string representation of datetime and returns a datetime object based on a format string.	`"date_parse_func": "{{date_parse('2020-01-01', '%Y-%m-%d')}}"`	`"date_parse_func": "2020-01-01 00:00:00"`
`datetime`	Python object to present the value in JSON for a given date and time. See Python datetime objects.	`"datetime_variable": "{{datetime(2020, 10, 28, 14, 30)}}"`	`"datetime_variable": "2020-10-28 14:30:00"`
`time`	Python object to present the value in JSON for a given time. See Python time objects.	`"time_variable": "{{time(14, 30)}}"`	`"time_variable": "14:30:00"`
`timedelta`	Python object to present the value in JSON for the difference between two dates or times. See Python timedelta objects.	`"timedelta_variable": "{{timedelta(weeks=2).days}}"`	`"timedelta_variable": "14"`

Convert types¶

Function name	Description
`int(val)`	Returns an integer from its string representation.
`float(val)`	Returns a float from its string representation.
`str(val)`	Returns a string representation of any value.
`bool(val)`	Returns a Python boolean, `True` or `False`, from its string representation. Use extra parameter for boolean function in JSON: The function `"some_key": {{bool(some_variable)}}` used in a JSON file compiles to `"some_key": True`, which is invalid JSON. Use the following syntax instead: `"some_key": {{bool(some_variable) \| lower}}`

Load files¶

Function name Description

load_csv(csv_file, delimiter=',')

Loads a .csv file from a path csv_file relative to the /resources directory in the recipe. Returns an array.

If the file has a single column, an array of strings is returned. If the file has multiple columns, an array of tuples is returned. Has an optional delimiter parameter that denotes the file's column delimiter.

load_text(file, escapejson=True, params={})

Loads a text (e.g. .sql) file from a path relative to the /resources directory in the recipe. These text files may include Jinja expressions.

Has an optional escapejson parameter, which defaults to true, escapes all newlines, tabs, and special characters to make the text suitable to be in a JSON string field. Has an optional params parameter; a dictionary of additional parameters to be used to process the file. These parameters override existing runtime variables.

Example keys: See Data Source SQL Fields for more information.

"sql": "{{load_text('filename_here.sql')}}"
"sql": "{{load_text('subfolder/filename_here.sql')}}"
"sql": "{{load_text('subfolder/filename_here.sql', escapejson=False, params={ 'key1': 'value1', 'key2': 'value2' })}}"
"sql": "{{load_text(sqlFilenameVariable)}}"
"filenamekey": "{{load_text('filename.sh')}}"
"schedule_process": "{{load_text('schedule_process.py')}}"

Parameters limitations: The load_text parameters are not supported through the SQL field Shared Recipe File view in the Node Editor. They can be entered through a Jinja expression using the File Editor.

load_json(file) Loads a .json file from the /resources directory in the recipe. Returns the specified file's contents as JSON.

Perform miscellaneous conversions¶

Function name	Description
`all(items)`	Returns true when all items of a list are true.
`any(items)`	Returns true when any of the items in the list is true.
`basename(path)`	Returns the last name of a file path, as in /etc/asdf.
`dict(some_dict_variable)`	Returns a copy of the specified dictionary.
`dirname(path)`	Returns the directory from a path, as in /etc/asdf.
`enumerate(list)`	Returns an iterator of tuples of (index, element) of an array.
`len(array or dict)`	Returns the length of an array or dictionary.
`list(somedict.keys())`	Returns a list. For example, `list` can be used to extract the keys of a JSON dictionary.
`map(items)`	Applies a specified function to each item and returns a list of results.
`path_join(...)`	Concatenates pieces of a file path, taking care of leading and trailing slashes.
`range(stop)` `range(start, stop, [step])`	Returns a list of integers from 0 to `stop`, or a list of values from `start` to `stop` using a given `step`.
`random()`	Returns a random float value between 0.0 and 1.0.
`strjoin(separator, items)`	Concatenates the items in the array and returns the concatenated string. Two syntaxes are supported: `strjoin(',','singleline')` evaluates to `singleline` `strjoin(',',['1','2','3'])` evaluates to `1,2,3`