FTP

FTP data sources and sinks are in the file-based category of I/O connectors.

Tool documentation

• FTP documentation

Connector type values

The "type": value to use in the source or sink JSON files.

Connector type	Value
Data source	DKDataSource_FTP
Data sink	DKDataSink_FTP

Connection properties

The properties to use when connecting to an FTP instance from Automation.

Field	Scope	Type	Required?	Description
hostname	source/sink	string	yes	A valid host name/IP address.
passive	source/sink	Boolean	no	Boolean flag that enables/disables passive mode. Deactivated by default.
password	source/sink	string	yes	FTP service password.
port	source/sink	string/number	yes	FTP port.
username	source/sink	string	yes	FTP service user name.

Connections

See Connection Properties for details on connection configurations

Defined in kitchen-level variables

ftpconfig in kitchen overrides

{
    "ftpconfig": {
        "passive": "#{vault://ftp/passive}",
        "username": "#{vault://ftp/username}",
        "password": "#{vault://ftp/password}",
        "hostname": "#{vault://ftp/hostname}",
        "port": "#{vault://ftp/port}"
    }
}

The Connection tab in a Node Editor

Expanded connection syntax

For a data source

ftp_datasource.json

{
    "type": "DKDataSource_FTP",
    "name": "ftp_datasource",
    "config": {
        "passive": "{{ftpconfig.passive}}",
        "username": "{{ftpconfig.username}}",
        "password": "{{ftpconfig.password}}",
        "hostname": "{{ftpconfig.hostname}}",
        "port": "{{ftpconfig.port}}"
    },
    "keys": {},
    "tests": {}
}

For a data sink

ftp_datasink.json

{
    "type": "DKDataSink_FTP",
    "name": "ftp_datasink",
    "config": {
        "passive": "{{ftpconfig.passive}}",
        "username": "{{ftpconfig.username}}",
        "password": "{{ftpconfig.password}}",
        "hostname": "{{ftpconfig.hostname}}",
        "port": "{{ftpconfig.port}}"
    },
    "keys": {},
    "tests": {}
}

Condensed connection syntax

For a data source

ftp_datasource.json

{
    "type": "DKDataSource_FTP",
    "name": "ftp_datasource",
    "config-ref": "ftpconfig",
    "keys": {},
    "tests": {}
}

For a data sink

ftp_datasink.json

{
    "type": "DKDataSink_FTP",
    "name": "ftp_datasink",
    "config-ref": "ftpconfig",
    "keys": {},
    "tests": {}
}

Local connections

FTP file contents may be viewed locally by configuring connections with file-transfer applications.

Other configuration properties

See the following topics for common properties, wildcards, and runtime variables:

• File-Based Properties
• File-Based Wildcards
• File-Based Variables

Note

Pre-existing path requirement. FTP data sinks can only PUT files to existing output directories. They cannot create subfolders during processing. Other file-based data sinks, such as S3 or Google Cloud Storage, have the underlying functionality to create folders automatically.

File encoding requirements

Files used with data sources and data sinks must be encoded in UTF-8 in order to avoid non-Unicode characters causing problems with sinking data to database tables and errors when running related tests

For CSV and other delimited files, use Save as in the program and select the proper encoding, or consider using a text editor with encoding options.

Data source example

ftp_datasource.json

{
    "type": "DKDataSource_FTP",
    "name": "ftp_datasource",
    "config-ref": "ftpconfig",
    "keys": {},
    "tests": {}
}

Data sink example

This FTP data sink pushes a CSV file to FTP via an explicit key and generates runtime variables for tests.

ftp_datasink.json

{
    "type": "DKDataSink_FTP",
    "name": "ftp_datasink",
    "config": {
        "username": "{{ftpconfig.username}}",
        "password": "{{ftpconfig.password}}",
        "hostname": "{{ftpconfig.host}}",
        "port": "{{ftpconfig.port}}"
    },
    "set-runtime-vars": {
        "key_count": "ftp_sink_key_count",
        "key_files": "ftp_sink_key_files",
        "key_names": "ftp_sink_key_names"
    },
    "keys": {
        "key1": {
            "file-key": "ftp-test/sample-recipe-datamapper-salesforce-ftp.csv",
            "use-only-file-key": true,
            "set-runtime-vars": {
                "md5": "ftp_sink_md5",
                "sha": "ftp_sink_sha",
                "size": "ftp_sink_size",
                "row_count": "ftp_sink_row_count"
            }

        }

    },
    "tests": {

    }
}