Skip to content

Test Definitions

View and manage the individual tests in a test suite — review auto-generated tests, tune parameters, add manual tests, and control which tests run during execution.

Test definitions sit between test generation (which auto-creates definitions from profiling results) and test execution (which runs them against your data). This is where you fine-tune what gets tested and how.

Prerequisites

A test suite with generated or manually created tests. To populate a test suite automatically, see Generate Tests.

  1. Select Test Suites from the menu.
  2. On a test suite, click View N test definitions to open the list of test definitions for that suite.

Note

On a contract, the tests are on the Quality Rules tab, and your changes are saved to the contract's draft. See Edit a contract's tests and document.

Review and filter tests

The test definitions grid displays all tests in the suite. Each row shows the test's table, column or focus area, test type, and status flags such as Active, Locked, and Flagged.

Use the filters at the top of the grid to narrow the list by Table, Column, Test Type, or Flagged status. Click any row to view the full test definition details in the bottom panel.

Edit a test definition

Adjust thresholds, expected values, or other parameters after reviewing test results.

  1. Select the test definition to open the detail panel.
  2. Click Edit.
  3. Modify the parameters as needed.
  4. Click Save.

Note

The test type, schema, and test suite cannot be changed after a test is created. To use a different test type, create a new test definition.

Add a test manually

Add a test of any type, including the types that test generation creates, and set any of its parameters, such as baseline values, thresholds, and tolerances.

  1. Click Add.
  2. Select a test type and fill in the required parameters.
  3. Click Add.

The Table and Column fields list the tables and columns in the table group's data catalog. To write a test ahead of the data, such as for a table a pipeline is about to create or a column being added, type the name, such as orders_v2, and choose Use "orders_v2" at the end of the list, or move on to the next field. Enter picks the first listed table or column that contains what you typed, and uses the name as typed when none does. A name that is not in the data catalog is flagged under its field, and you can still save the test. Until the table or column exists, the test fails validation on each test run.

See TestGen Test Types for the full list of available test types and their descriptions.

Tip

To add a Custom Condition, SQL Metric, or Custom Test, see Add Custom SQL Tests.

Test only some records

To apply a test to only some records, such as "ship date is required when status is SHIPPED", set its Record Subset Condition. Left blank, the test covers every record.

  1. Add or edit a test.
  2. In Record Subset Condition, enter a condition written like a SQL WHERE clause, such as status = 'SHIPPED'.
  3. Click Add or Save.

Duplicate Rows, Table Comparison tests, and tests written in SQL (Custom Condition, SQL Metric, and Custom Test) have no Record Subset Condition. Filter a SQL test in its own query.

Note

Regenerating tests clears the condition on an unlocked auto-generated test. Lock the test to keep it.

Lock and unlock tests

Lock a test to protect its manually tuned parameters from being overwritten when you regenerate tests. Unlocked auto-generated tests are refreshed during regeneration — their parameters may change to reflect current data.

Select one or more tests and change the Locked status using the (lock icon) and (unlock icon).

Note

Manually created tests are never affected by regeneration, so they do not need to be locked.

Activate and deactivate tests

Deactivate a test to exclude it from future test runs without deleting it. This is useful for temporarily disabling tests that produce noisy results while you investigate.

Select one or more tests and toggle the Active status using the (activate icon) and (deactivate icon). Only active tests run during test execution.

Copy or move tests

Reuse a tuned test across suites or tables by copying or moving it.

  1. Select one or more test definitions.
  2. Click the (copy/move icon).
  3. Choose the target test suite and table.
  4. Click Copy or Move.

The target can be any test suite, including a contract. Tests copied or moved into a contract are saved to its draft, and TestGen creates a draft when the contract has none. Each test must be on a table the contract declares. Moving tests out of a contract removes them from its draft. Schema Conformance tests are not copied or moved.

Flag tests for review

You can flag test definitions to mark them for attention using the (flag icon) and (clear flag icon). Flagged tests can be filtered using the Flagged dropdown on both the Test Definitions and Test Results pages. See Investigate Test Results for details.

Add notes to a test

You can add timestamped notes to any test definition to document investigation context, decisions, or open questions. Select a test definition to open the detail panel, and click Notes to view or add notes. See Investigate Test Results for details. On a contract, a test's notes are shared by every version of that test.

You can add the same note to several tests at once.

  1. Select one or more tests.
  2. Click the (add note icon).
  3. In Note, enter the note.
  4. Click Add Note.

Each selected test gets its own copy of the note. Editing or deleting one copy leaves the others unchanged.

Export test definitions

Click Export to download test definitions.

  • Under Excel, export all definitions, only the definitions matching your current filters, or only selected definitions, for review.
  • Under JSON, select Test definitions to download every definition in the suite as a file you can edit and import into this test suite or another one.

Import test definitions

Import a JSON file of test definitions to update tests in bulk, copy tests from another test suite, or transfer tests to another table group, as in Verify a Data Migration.

  1. Click Import.
  2. Upload the test definitions JSON file.
  3. Choose what happens to Tests in the file with no match, Tests in the file that match, Tests not in the file, and Tests for tables or columns not in the data catalog.
  4. Review the preview of what will be created, updated, skipped, and deleted. If tests will be deleted, confirm the deletion. Click Import Tests.

Only the fields present in the file are written. A matching test keeps its values for any field the file leaves out.

An auto-generated test matches a test of the same type on the same table and column. A manual test matches by its external_id, so import skips a manual test without one. Export assigns an external_id to every manual test. Set one yourself in a file you write from scratch. By default, import also skips a test whose table is not in the table group's data catalog, and a column-scoped test whose column is not in that table. To import those tests ahead of the data, set Tests for tables or columns not in the data catalog to Import. The preview still names them as Table not in the data catalog or Column not in the data catalog, with the action Create or Update.

Tip

You can also export and import test definitions through the REST API. See Import and Export Tests for details.

Run tests

Click Run Tests to run the active tests in the suite. For more options, including scheduling, see Run Tests.