Skip to content

Connect your Data

Before you can profile and test your data, you need to connect TestGen to it and tell it which tables to analyze.

TestGen reads two kinds of data sources:

  • Databases — TestGen queries your tables in place, through a read-only database user.
  • File data sources — TestGen reads CSV, Parquet, and JSON files, and you name each file or pattern of files that you want treated as a table.

Both are configured the same way, in three layers:

  • Connection — where your data lives and what TestGen needs to reach it. For a database, the server details and credentials. For files, the location of the files. See Manage Connections.
  • Table group — the tables to profile, test, and monitor together. For a database, a schema and a filter over its tables. For files, a declared list of tables. Each connection can have multiple table groups. See Manage Table Groups.
  • Test suite — a collection of data quality tests for a table group. See Manage Test Suites.

See Access Requirements for what TestGen needs from each supported source.

Tip

The Data Configuration Setup wizard (available when adding a connection) walks you through creating a connection, table group, and optionally setting up profiling, testing, and monitors in one flow.

The open-source version is limited to a single connection. Select Connection from the menu to edit its configuration, then click Setup Table Groups to create a table group.