Skip to content

Properties and Variables

The available configuration fields, required properties, optional properties, and runtime variables are specific to the data source or data sink type (file-based or database). In some cases, the properties and variables are specific to the I/O connector itself.

For example, the column_count variable is only supported for keys in database source or sink configurations, while the retry-count property is only used for SFTP sources and sinks.

Resources

This documentation provides an overview of the properties, fields, variables, and values to configure for data source and sink use.

  • Connection Properties: the configuration properties required for connections to data sources and data sinks.
  • Built-In Runtime Variables: sources and sinks provide several built-in variables available at runtime and useful for tests.
  • File-Based Properties: required and optional configuration properties for file-based sources/sinks and their key definitions.
  • File-Based Variables: optional runtime variables available for file-based sources/sinks.
  • File-Based Wildcards: file-based data sources and sinks allow identifying arbitrary files using wildcards.
  • Database Properties: required and optional configuration properties for database sources/sinks and their key definitions.
  • Database Variables: optional runtime variables available for database sources/sinks.