Consistency Tests¶
Consistency tests verify that data stays aligned with its baseline, with related tables, and with earlier time periods. They detect drift in statistical measures, any change from an exact baseline, totals or distributions that disagree with a reference table, and values that appear or disappear between time windows — catching problems that emerge when data stops agreeing with itself or with prior known-good states.
Average Shift¶
Tests for a statistically significant shift in the column mean compared to the average calculated at baseline.
Auto-generated for numeric measurement columns with more than 10 distinct values.
| Scope | Column |
| Measures | Cohen's D Difference (0.20 = small, 0.50 = moderate, 0.80 = large, 1.20 = very large) |
| Threshold | Maximum acceptable Cohen's D Difference (default: 0.5) |
| On failure | Standardized difference between averages is over the selected threshold level. |
| Default severity | Warning |
How it works: Computes the pooled Cohen's D statistic — the standardized difference between the current column average and the baseline average, normalized by their combined standard deviation. The test fails when this difference exceeds the threshold.
When to use: Average Shift is useful for any numeric measure where you expect the mean to remain stable over time. A significant shift may indicate errors in processing, differences in source data, or valid changes that could impact downstream assumptions. You may want to periodically adjust the baseline average if legitimate drift is expected.
Match Row Count¶
Tests that the number of rows in the table equals the baseline row count.
Auto-generated by the Table Comparison generation set for every table.
| Scope | Table |
| Measures | Row count |
| Threshold | Expected row count (default: count at baseline) |
| On failure | Row count differs from baseline. |
| Default severity | Fail |
How it works: Counts the rows in the table. The test fails when the count is higher or lower than the expected count.
When to use: Use this test to confirm that a copy or a reload has the same row count as its original, or that a reference table has not gained or lost rows. For a minimum rather than an exact count, use Row Count.
Match Column Count¶
Tests that the number of columns in the table equals the baseline column count.
Auto-generated by the Table Comparison generation set for every table.
| Scope | Table |
| Measures | Column count |
| Threshold | Expected column count (default: count at baseline) |
| On failure | Column count differs from baseline. |
| Default severity | Fail |
How it works: Counts the columns that the table has when the test runs. The test fails when the count differs from the expected count.
When to use: A failure means columns were added to or removed from the table. Pair it with Match Column Type to also catch columns whose type changed.
Match Column Type¶
Tests that the general data type of the column equals the baseline type.
Auto-generated by the Table Comparison generation set for every profiled column.
| Scope | Column |
| Measures | Whether the type matches baseline |
| Threshold | Expected general type (default: type at baseline) |
| On failure | Column general type differs from baseline. |
| Default severity | Fail |
How it works: Compares the general type of the column when the test runs to the expected type: A for text, N for numeric, D for date or timestamp, T for time of day, B for boolean, or X for any other type. The test fails when the two differ.
When to use: Use this test to catch a column that was redefined as a different kind of data, such as a date column loaded as text. A change of specific type within the same general type, such as a wider text column, does not fail it.
Match Null Count¶
Tests that the number of null values in the column equals the baseline count.
Auto-generated by the Table Comparison generation set for every column in a table with at least one row.
| Scope | Column |
| Measures | Null values |
| Threshold | Expected null count (default: count at baseline) |
| On failure | Null count differs from baseline. |
| Default severity | Fail |
How it works: Counts the null values in the column. The test fails when the count differs from the expected count.
When to use: A change points to values lost or filled in during a load. Read it alongside Match Zero Count and Match Empty Strings: a drop in nulls with a matching rise in zeros or empty strings means nulls were replaced.
Match Distinct Count¶
Tests that the number of distinct values in the column equals the baseline count.
Auto-generated by the Table Comparison generation set for every column of a text, numeric, date, time, or boolean type in a table with at least one row.
| Scope | Column |
| Measures | Unique values |
| Threshold | Expected distinct value count (default: count at baseline) |
| On failure | Distinct value count differs from baseline. |
| Default severity | Fail |
How it works: Counts the distinct non-null values in the column. The test fails when the count differs from the expected count.
When to use: Use this test where the column should hold exactly as many distinct values as it held at baseline. To detect any change to the values themselves, use Match Distinct Hash. For a count that is only expected not to drop, use Value Count.
Match Minimum Value¶
Tests that the lowest value in the column equals the baseline minimum.
Auto-generated by the Table Comparison generation set for numeric columns with at least one value.
| Scope | Column |
| Measures | Minimum value |
| Threshold | Expected minimum value (default: minimum at baseline) |
| On failure | Minimum value differs from baseline. |
| Default severity | Fail |
How it works: Finds the lowest value in the column and compares it to the expected value as a number. The test fails when the two differ.
When to use: Use this test where the lowest value should stay exactly as it was. For a floor that values may stay above, use Minimum Value.
Match Maximum Value¶
Tests that the highest value in the column equals the baseline maximum.
Auto-generated by the Table Comparison generation set for numeric columns with at least one value.
| Scope | Column |
| Measures | Maximum value |
| Threshold | Expected maximum value (default: maximum at baseline) |
| On failure | Maximum value differs from baseline. |
| Default severity | Fail |
How it works: Finds the highest value in the column and compares it to the expected value as a number. The test fails when the two differ.
When to use: Use this test where the highest value should stay exactly as it was. For a ceiling that values may stay below, use Maximum Value.
Match Zero Count¶
Tests that the number of zero values in the column equals the baseline count.
Auto-generated by the Table Comparison generation set for numeric columns with at least one value.
| Scope | Column |
| Measures | Zero values |
| Threshold | Expected zero value count (default: count at baseline) |
| On failure | Zero value count differs from baseline. |
| Default severity | Fail |
How it works: Counts the values in the column that equal zero. The test fails when the count differs from the expected count.
When to use: A change points to zeros added or removed during a load. Read it with Match Null Count.
Match Sum¶
Tests that the column total equals the baseline total, within a relative tolerance.
Auto-generated by the Table Comparison generation set for numeric columns with at least one value and a total not close to zero.
| Scope | Column |
| Measures | Column sum |
| Threshold | Expected column sum (default: sum at baseline) and relative tolerance (default: 0 for most integer and exact-decimal columns, 0.000000001 otherwise) |
| On failure | Column sum differs from baseline by more than the tolerance. |
| Default severity | Fail |
How it works: Sums the values in the column and compares the total to the expected sum. The test fails when the difference is larger than the expected sum times the relative tolerance. When the tolerance is not 0, the test also allows 0.000000005 for each non-null value, for rounding to eight decimal places. A tolerance of 0 requires an exact match. A tolerance of 0.001 allows a difference of 0.1 percent of the expected sum, plus that allowance.
When to use: Widen the tolerance per test when small numeric differences are expected, for example between databases that store decimals with different precision. Pair it with Match Standard Deviation.
Match Standard Deviation¶
Tests that the standard deviation of the column equals the baseline, within a relative tolerance.
Auto-generated by the Table Comparison generation set for numeric columns with at least two values.
| Scope | Column |
| Measures | Standard deviation |
| Threshold | Expected standard deviation (default: value at baseline), and the allowed difference as a fraction of it (default: 0.000000001) |
| On failure | Standard deviation differs from baseline by more than the tolerance. |
| Default severity | Fail |
How it works: Computes the sample standard deviation of the column and compares it to the expected value. The test fails when the difference is larger than the relative tolerance times the expected value.
When to use: Together with Match Sum, this test detects offsetting changes that leave the total unchanged, such as one value raised and another lowered by the same amount.
Match Fractional Sum¶
Tests that the sum of the fractional parts of the column values equals the baseline sum.
Auto-generated by the Table Comparison generation set for numeric columns.
| Scope | Column |
| Measures | Fractional sum |
| Threshold | Expected fractional sum (default: sum at baseline) |
| On failure | Sum of fractional parts differs from baseline. |
| Default severity | Fail |
How it works: Takes the fractional part of each value (the portion after the decimal point), rounds it to five decimal places, and sums the results. The test fails when the sum differs from the expected sum.
When to use: A drop points to values rounded or truncated to fewer decimal places. For a floor rather than an exact sum, use Decimal Truncation.
Match Minimum Date¶
Tests that the earliest date in the column equals the baseline date, compared to the second.
Auto-generated by the Table Comparison generation set for date columns with at least one value.
| Scope | Column |
| Measures | Earliest date |
| Threshold | Expected earliest date, in the form YYYY-MM-DD HH:MM:SS (default: earliest date at baseline) |
| On failure | Earliest date differs from baseline. |
| Default severity | Fail |
How it works: Finds the earliest date in the column and compares it to the expected date. Fractional seconds are ignored.
When to use: Use this test where the earliest date should stay exactly as it was. For a floor that dates may stay above, use Minimum Date.
Match Maximum Date¶
Tests that the latest date in the column equals the baseline date, compared to the second.
Auto-generated by the Table Comparison generation set for date columns with at least one value.
| Scope | Column |
| Measures | Latest date |
| Threshold | Expected latest date, in the form YYYY-MM-DD HH:MM:SS (default: latest date at baseline) |
| On failure | Latest date differs from baseline. |
| Default severity | Fail |
How it works: Finds the latest date in the column and compares it to the expected date. Fractional seconds are ignored.
When to use: Use this test where the latest date should stay exactly as it was, such as on a copy of a closed historical range. In a table that is still being loaded, the latest date moves forward with every load and this test fails. For a ceiling that dates may stay below, use Maximum Date.
Match Date Count¶
Tests that the number of distinct calendar days in the column equals the baseline count.
Auto-generated by the Table Comparison generation set for date columns.
| Scope | Column |
| Measures | Unique dates |
| Threshold | Expected distinct date count (default: count at baseline) |
| On failure | Count of distinct dates differs from baseline. |
| Default severity | Fail |
How it works: Counts the distinct calendar days present in the column, ignoring the time of day. The test fails when the count differs from the expected count.
When to use: A change with an unchanged row count often points to a time zone shift that moved values across midnight. For a count that is only expected not to drop, use Date Count.
Match True Count¶
Tests that the number of true values in the column equals the baseline count.
Auto-generated by the Table Comparison generation set for boolean columns with at least one value.
| Scope | Column |
| Measures | True values |
| Threshold | Expected true value count (default: count at baseline) |
| On failure | True value count differs from baseline. |
| Default severity | Fail |
How it works: Counts the true values in the column. The test fails when the count differs from the expected count.
When to use: With Match Null Count and Match Row Count, this test accounts for every value in a boolean column.
Match Minimum Length¶
Tests that the length of the shortest value in the column equals the baseline length.
Auto-generated by the Table Comparison generation set for text columns with at least one value.
| Scope | Column |
| Measures | Minimum length |
| Threshold | Expected minimum length in characters (default: length at baseline) |
| On failure | Minimum text length differs from baseline. |
| Default severity | Fail |
How it works: Finds the length in characters of the shortest non-null value in the column. The test fails when the length differs from the expected length.
When to use: A drop points to values shortened or blanked during a load.
Match Maximum Length¶
Tests that the length of the longest value in the column equals the baseline length.
Auto-generated by the Table Comparison generation set for text columns with at least one value.
| Scope | Column |
| Measures | Maximum length |
| Threshold | Expected maximum length in characters (default: length at baseline) |
| On failure | Maximum text length differs from baseline. |
| Default severity | Fail |
How it works: Finds the length in characters of the longest value in the column. The test fails when the length differs from the expected length.
When to use: A drop points to truncation, such as values loaded into a narrower column. For a floor rather than an exact length, use Alpha Truncation.
Match Total Length¶
Tests that the combined length of all values in the column equals the baseline total.
Auto-generated by the Table Comparison generation set for text columns with at least one value.
| Scope | Column |
| Measures | Total length |
| Threshold | Expected total length in characters (default: total at baseline) |
| On failure | Total text length differs from baseline. |
| Default severity | Fail |
How it works: Adds up the length in characters of every non-null value in the column. The test fails when the total differs from the expected total.
When to use: A change points to values truncated, padded, trimmed, or rewritten with a different length, such as characters damaged by an encoding conversion. Pair it with Match Minimum Length and Match Maximum Length: it also catches changes that leave the shortest and longest values as they were. It does not detect a change that keeps each value's length, such as one letter replaced by another.
Match Leading Spaces¶
Tests that the number of values with leading spaces equals the baseline count.
Auto-generated by the Table Comparison generation set for text columns with at least one value.
| Scope | Column |
| Measures | Values with leading spaces |
| Threshold | Expected leading space count (default: count at baseline) |
| On failure | Count of values with leading spaces differs from baseline. |
| Default severity | Fail |
How it works: Counts the values in the column that begin with a space. The test fails when the count differs from the expected count.
When to use: A drop points to values trimmed during a load.
Match Empty Strings¶
Tests that the number of empty strings in the column equals the baseline count.
Auto-generated by the Table Comparison generation set for text columns with at least one value.
| Scope | Column |
| Measures | Empty strings |
| Threshold | Expected empty string count (default: count at baseline) |
| On failure | Empty string count differs from baseline. |
| Default severity | Fail |
How it works: Counts the values in the column that are empty strings. Null values are not counted. The test fails when the count differs from the expected count.
When to use: A change points to empty strings added or removed during a load. Read it with Match Null Count.
Match Upper Case¶
Tests that the number of all-upper-case values in the column equals the baseline count.
Auto-generated by the Table Comparison generation set for text columns with at least one value.
| Scope | Column |
| Measures | Upper-case values |
| Threshold | Expected upper-case count (default: count at baseline) |
| On failure | Upper-case value count differs from baseline. |
| Default severity | Fail |
How it works: Counts the values in the column that contain at least one letter and no lower-case letters. The test fails when the count differs from the expected count.
When to use: A change points to values re-cased during a load.
Match Non-Printing¶
Tests that the number of values containing non-printing characters equals the baseline count.
Auto-generated by the Table Comparison generation set for text columns with at least one value.
| Scope | Column |
| Measures | Values with non-printing characters |
| Threshold | Expected non-printing count (default: count at baseline) |
| On failure | Count of values with non-printing characters differs from baseline. |
| Default severity | Fail |
How it works: Counts the values in the column that contain a non-printing character, such as a non-breaking space or a zero-width space. The test fails when the count differs from the expected count.
When to use: A change points to encoding damage, or to cleanup during a load.
Match Distinct Hash¶
Tests that the set of distinct values in the column equals the baseline set, using a hash of the values.
Auto-generated by the Table Comparison generation set for text columns with 2 to 70 distinct values of up to 70 characters.
| Scope | Column |
| Measures | Whether the hash matches baseline |
| Threshold | Expected distinct value hash (default: hash at baseline) |
| On failure | Set of distinct values differs from baseline. |
| Default severity | Fail |
How it works: Sorts the distinct non-null values in the column by their bytes, joins them with the pipe character (|), and computes the MD5 hash of the result. The test fails when the hash differs from the expected hash.
When to use: Values are compared exactly, including case and spacing, so this test detects any change to the set of values in a text column. A failure says the set changed, not how. For a text column with more distinct values, or to also catch a change in how often a value occurs, use Match Text Hash. Match Frequent Values shows which of the most common values differ, and the source data for a failed test lists the distinct values with their counts.
Match Frequent Values¶
Tests that the ten most frequent values in the column, and the count of each, equal the baseline.
Auto-generated by the Table Comparison generation set for text columns whose ten most frequent values, written out, fit in 1,000 characters.
| Scope | Column |
| Measures | Whether the values match baseline |
| Threshold | Expected frequent values (default: values at baseline) |
| On failure | Most frequent values or their counts differ from baseline. |
| Default severity | Fail |
How it works: Ranks the non-blank values in the column by how often they occur, with values of equal count listed in byte order. The ten highest are written as value (count), separated by a comma and a space, for example Open (120), Closed (45). The test fails when the result differs from the expected list.
When to use: This is the most direct test of how the distribution of a column has moved. A failure shows both which values and which counts changed.
Match Frequent Patterns¶
Tests that the five most frequent character patterns in the column, and the count of each, equal the baseline.
Auto-generated by the Table Comparison generation set for text columns.
| Scope | Column |
| Measures | Whether the patterns match baseline |
| Threshold | Expected frequent patterns (default: patterns at baseline) |
| On failure | Most frequent patterns or their counts differ from baseline. |
| Default severity | Fail |
How it works: Represents each non-blank value by its character classes: each lower-case letter becomes a, each upper-case letter A, and each digit N. Other characters stay as they are. The five most frequent patterns, with patterns of equal count listed in byte order, are written as pattern (count), separated by a comma and a space, for example AA-NNNN (230), AA-NNN (12). The test fails when the result differs from the expected list.
When to use: A failure points to a change in the shape of values, such as a new format for codes.
Match Text Hash¶
Tests that the text values in the column, and how often each occurs, equal the baseline, by comparing a sum of per-value hashes.
Auto-generated by the Table Comparison generation set for text columns. Requires the table group's Profile for table comparison setting.
| Scope | Column |
| Measures | Hash sum |
| Threshold | Expected hash sum (default: sum at baseline) |
| On failure | Text values differ from baseline. |
| Default severity | Fail |
How it works: Hashes each non-null value and adds up the hashes. Text is compared as stored, except that on SQL Server, PostgreSQL, and Redshift the spaces that pad a fixed-length column are ignored. On SQL Server, text values are hashed only where the server supports UTF-8 collations, such as SQL Server 2019 and later and Azure SQL Database. On other SQL Server versions, no text hash tests, keyed or unkeyed, are generated, and a text column cannot be a table's key. The test fails when the sum differs from the expected sum.
When to use: This test detects a changed, added, or dropped value in any row order. It covers columns with too many distinct values for Match Distinct Hash. A failure says the values differ, not how; the column's counts, lengths, and frequent values show what changed.
Match Numeric Hash¶
Tests that the numeric values in the column, and how often each occurs, equal the baseline, by comparing a sum of per-value hashes.
Auto-generated by the Table Comparison generation set for numeric columns. Requires the table group's Profile for table comparison setting.
| Scope | Column |
| Measures | Hash sum |
| Threshold | Expected hash sum (default: sum at baseline) |
| On failure | Numeric values differ from baseline. |
| Default severity | Fail |
How it works: Hashes each non-null value and adds up the hashes. A number is rounded half away from zero to six decimal places, and a floating-point value is first rounded to nine. A floating-point value hashes like the same decimal value when the decimal has up to nine decimal places. Values that round to 1032 or more in magnitude (1029 for floating-point values), infinity, and not-a-number are left out of the hash. The test fails when the sum differs from the expected sum.
When to use: This test detects a changed, added, or dropped value in any row order. A failure says the values differ, not how; the column's counts, extremes, and sums show what changed.
Match Date Hash¶
Tests that the date values in the column, and how often each occurs, equal the baseline, by comparing a sum of per-value hashes.
Auto-generated by the Table Comparison generation set for date and timestamp columns. Requires the table group's Profile for table comparison setting.
| Scope | Column |
| Measures | Hash sum |
| Threshold | Expected hash sum (default: sum at baseline) |
| On failure | Date values differ from baseline. |
| Default severity | Fail |
How it works: Hashes each non-null value and adds up the hashes. A date or timestamp is compared to the second, in UTC when it carries a time zone, and a date reads as midnight. Infinite dates are left out of the hash. The test fails when the sum differs from the expected sum.
When to use: This test detects a changed, added, or dropped value in any row order. A failure says the values differ, not how; the column's extremes, date count, and Match Epoch Sum show what changed.
Match Epoch Sum¶
Tests that the sum of the column's dates, as seconds since 1970, equals the baseline sum.
Auto-generated by the Table Comparison generation set for date and timestamp columns. Requires the table group's Profile for table comparison setting.
| Scope | Column |
| Measures | Epoch seconds |
| Threshold | Expected epoch sum (default: sum at baseline) |
| On failure | Sum of dates differs from baseline. |
| Default severity | Fail |
How it works: Converts each non-null date or timestamp to whole seconds since 1970-01-01 UTC, truncating any fraction of a second, and adds them up. Infinite dates are left out of the sum. The test fails when the sum differs from the expected sum.
When to use: A uniform time zone shift moves the sum by the number of non-null values times the offset in seconds. Divide the difference by that number to read the shift. Changes that offset each other cancel in the sum, so read it with Match Date Hash.
Match Keyed Text Hash¶
Tests that each text value in the column is attached to the same row as at baseline, by comparing a sum of hashes of each row's key and value.
Auto-generated by the Table Comparison generation set for each text column of a table with a key column, other than the key. Requires the table group's Profile for table comparison setting.
| Scope | Column |
| Measures | Keyed hash sum |
| Threshold | Expected keyed hash sum (default: sum at baseline) |
| On failure | Text values are not attached to the same rows as at baseline. |
| Default severity | Fail |
How it works: Hashes each row's key together with its value and adds up the hashes. The test's Key Column parameter names the key. Profiling picks the key from the columns with a distinct, non-null value in every row. Floating-point columns, numeric columns with more than six decimal places, date and time columns other than DATE, and text longer than 100 characters are not candidates. It prefers an ID column, ranking a surrogate key after the other ID columns, then whole numbers, dates, text, and other numbers. The key and the value are each compared as in Match Text Hash. The test fails when the sum differs from the expected sum.
When to use: This test detects values moved between rows, which no unkeyed test detects. Read it with Match Text Hash: when that test passes and the keyed hash fails, the same values are attached to different rows. If every keyed hash test on a table fails while the unkeyed hash tests pass, the key itself changed, such as a surrogate key regenerated by a reload.
Match Keyed Numeric Hash¶
Tests that each numeric value in the column is attached to the same row as at baseline, by comparing a sum of hashes of each row's key and value.
Auto-generated by the Table Comparison generation set for each numeric column of a table with a key column, other than the key. Requires the table group's Profile for table comparison setting.
| Scope | Column |
| Measures | Keyed hash sum |
| Threshold | Expected keyed hash sum (default: sum at baseline) |
| On failure | Numeric values are not attached to the same rows as at baseline. |
| Default severity | Fail |
How it works: Hashes each row's key together with its value and adds up the hashes. The test's Key Column parameter names the key, which profiling picks as described in Match Keyed Text Hash. The key and the value are each compared as in Match Numeric Hash. The test fails when the sum differs from the expected sum.
When to use: This test detects values moved between rows, which no unkeyed test detects. Read it with Match Numeric Hash: when that test passes and the keyed hash fails, the same values are attached to different rows.
Match Keyed Date Hash¶
Tests that each date value in the column is attached to the same row as at baseline, by comparing a sum of hashes of each row's key and value.
Auto-generated by the Table Comparison generation set for each date and timestamp column of a table with a key column, other than the key. Requires the table group's Profile for table comparison setting.
| Scope | Column |
| Measures | Keyed hash sum |
| Threshold | Expected keyed hash sum (default: sum at baseline) |
| On failure | Date values are not attached to the same rows as at baseline. |
| Default severity | Fail |
How it works: Hashes each row's key together with its value and adds up the hashes. The test's Key Column parameter names the key, which profiling picks as described in Match Keyed Text Hash. The key and the value are each compared as in Match Date Hash. The test fails when the sum differs from the expected sum.
When to use: This test detects values moved between rows, which no unkeyed test detects. Read it with Match Date Hash: when that test passes and the keyed hash fails, the same values are attached to different rows.
Aggregate Balance¶
Tests that aggregate values per grouping match exactly between the source table and a reference table.
This test is not auto-generated. You can create it manually from the Test Definitions page.
| Scope | Referential |
| Measures | Count of mismatched group totals |
| Threshold | Expected count of group totals not matching aggregate value (default: 0) |
| On failure | Aggregate measure per set of column values does not exactly match reference dataset. |
| Default severity | Fail |
How it works: Rolls up an aggregate expression (such as SUM or COUNT) by grouping columns in both the source and reference tables, then compares the totals for each group. Any group where the values differ — or where a group exists in one table but not the other — counts as a mismatch.
When to use: Use this test to verify that two tables produce identical aggregate results — for example, confirming that a summary table exactly matches its source detail table. Because the comparison is exact, even small rounding differences or new categories will trigger a failure.
Aggregate Balance Percent¶
Tests that aggregate values per grouping fall within a percentage range of the reference table values.
This test is not auto-generated. You can create it manually from the Test Definitions page.
| Scope | Referential |
| Measures | Count of mismatched group totals |
| Threshold | Expected count of group totals outside percentage range (default: 0) |
| On failure | Aggregate measure per set of column values is outside percent range of reference dataset. |
| Default severity | Fail |
How it works: Rolls up an aggregate expression by grouping columns in both tables, then checks whether each group's value in the source table falls within a lower and upper percentage tolerance of the corresponding reference value. Groups outside this range count as mismatches.
When to use: Use this test when you expect two datasets to be similar but not identical — for example, comparing sales by product between two time periods. This is more forgiving than Aggregate Balance while still catching significant deviations.
Aggregate Balance Range¶
Tests that aggregate values per grouping fall within a fixed numeric range of the reference table values.
This test is not auto-generated. You can create it manually from the Test Definitions page.
| Scope | Referential |
| Measures | Count of mismatched group totals |
| Threshold | Expected count of group totals outside numeric range (default: 0) |
| On failure | Aggregate measure per set of column values is outside expected range of reference dataset. |
| Default severity | Fail |
How it works: Rolls up an aggregate expression by grouping columns in both tables, then checks whether each group's value in the source table falls within a fixed lower and upper tolerance of the reference value. The tolerances are expressed as constant values, not percentages.
When to use: Use this test when acceptable variation is best expressed as a fixed amount rather than a percentage — for example, when comparing weekly totals where a difference of up to 10,000 units is acceptable regardless of the baseline size.
Distribution Shift¶
Tests that the probability distribution of values across categories is consistent with a reference dataset.
This test is not auto-generated. You can create it manually from the Test Definitions page.
| Scope | Referential |
| Measures | Divergence level (0 = identical, 1 = maximum divergence) |
| Threshold | Expected maximum divergence level between 0 and 1 |
| On failure | Divergence between two distributions exceeds specified threshold. |
| Default severity | Warning |
How it works: Counts records per category in both the source and reference tables, converts the counts to proportional distributions, and measures the difference using Jensen-Shannon divergence. A score of 0 means the distributions are identical; a score of 1 means they share no common structure. The test fails when the divergence exceeds the threshold.
When to use: Use this test when two datasets should have a similar spread across categories but are not expected to match exactly. The divergence score is also useful as a stability metric over time — even if two distributions are never identical, you would expect a consistent divergence score from period to period.
Timeframe No Drops¶
Tests that column values present in the most recent time window include at least all values from the prior time window of the same duration.
This test is not auto-generated. You can create it manually from the Test Definitions page.
| Scope | Referential |
| Measures | Count of missing value combinations |
| Threshold | Expected count of missing value combinations (default: 0) |
| On failure | Column values in most recent time window do not include all values in prior time window. |
| Default severity | Fail |
How it works: Divides a table into two consecutive time windows based on a date column and a window length in days. Collects the distinct values (or value combinations) from each window and identifies any that appear in the earlier window but not the later one. The test fails when the count of dropped values exceeds the threshold.
When to use: Use this test to confirm that codes or categories are not lost across successive time periods in a transactional table. Missing values in the latest window trigger a failure; new values are permitted. Specify one or more categorical columns to compare — avoid using continuous measurements.
Timeframe Match¶
Tests that column values in the most recent time window exactly match those in the prior time window of the same duration.
This test is not auto-generated. You can create it manually from the Test Definitions page.
| Scope | Referential |
| Measures | Count of mismatched value combinations |
| Threshold | Expected count of non-matching value combinations (default: 0) |
| On failure | Column values do not match in most recent time windows. |
| Default severity | Fail |
How it works: Divides a table into two consecutive time windows based on a date column and a window length in days. Collects the distinct values (or value combinations) from each window and identifies any differences — values added or removed. The test fails when the count of mismatched values exceeds the threshold.
When to use: This is a stricter form of Timeframe No Drops. Both new and missing values in the latest window trigger a failure. Use this to confirm the consistency of codes or categories across successive time periods in a transactional table.