Timeliness Tests¶
Timeliness tests verify that data arrives on schedule and reflects expected time ranges. They detect stale tables, late deliveries, missing date coverage, and unexpected future dates — problems that can silently invalidate downstream reporting even when the data itself looks structurally correct.
Recency¶
Tests that the most recent date in the column is within a set number of days of the test date.
Auto-generated when profiling finds a date column whose maximum value is within 62 days of the test run date.
| Scope | Column |
| Measures | Days before test (number of days the most recent date precedes the test date) |
| Threshold | Maximum acceptable days before test (default: calculated from profiled recency) |
| On failure | Most recent date value not within expected days of test date. |
| Default severity | Warning |
How it works: Finds the maximum date value in the column and calculates how many days it precedes the test run date. The test fails when this day count exceeds the threshold.
When to use: This test evaluates recency based on the latest referenced dates in the column. It is appropriate for transactional dates and timestamps. The test is especially valuable because timely data deliveries themselves may not assure that the most recent data is present. Adjust the threshold to the maximum number of days you expect the data to age before the dataset is refreshed.
Date Count¶
Tests that the count of distinct dates referenced in the column has not dropped compared to baseline data.
Auto-generated when profiling finds a transactional date column with more than one distinct date in a cumulative table.
| Scope | Column |
| Measures | Count of unique dates in the column |
| Threshold | Minimum distinct date count expected (default: count at baseline) |
| On failure | Drop in count of unique dates recorded in column. |
| Default severity | Fail |
How it works: Counts the distinct date values present in the column and compares the result to the threshold. The test fails when the current count falls below the expected minimum.
When to use: This test is relevant for cumulative datasets where old records are retained and the number of distinct dates should only grow. A failure would indicate missing records, which could be caused by a processing error or changed upstream data sources.
Past Dates¶
Tests that the maximum date referenced in the column is no greater than the test date, consistent with baseline data.
Auto-generated when profiling finds a date column with no future dates at baseline.
| Scope | Column |
| Measures | Count of future dates |
| Threshold | Expected count of future dates (default: 0) |
| On failure | Future date found when absent in baseline data. |
| Default severity | Fail |
How it works: Counts date values in the column that fall after the test run date. The test fails when the count exceeds the threshold.
When to use: This test is appropriate for columns where dates should never extend beyond the current date, such as event timestamps or historical transaction dates. Unexpected future dates could indicate data entry errors or placeholder values.
Future Year¶
Tests that the maximum date referenced in the column is no greater than one year beyond the test date, consistent with baseline data.
Auto-generated when profiling finds a date column with future dates, all within one year of the test date.
| Scope | Column |
| Measures | Count of dates beyond one year in the future |
| Threshold | Expected count of dates beyond one year (default: 0) |
| On failure | Future date beyond one year found when absent in baseline. |
| Default severity | Fail |
How it works: Counts date values in the column that fall more than one year after the test run date. The test fails when the count exceeds the threshold.
When to use: This test is appropriate for columns where you expect near-future dates but not beyond one year. Errors could indicate invalid entries or possibly dummy dates representing blank values.
Table Freshness¶
Tests whether a table has been updated by comparing data fingerprints across runs.
Auto-generated based on profiling results.
| Scope | Table |
| Measures | Change detected (1 = updated, 0 = stale) |
| Threshold | Most recent prior table fingerprint |
| On failure | Table has not been updated. |
| Default severity | Log |
How it works: Computes a fingerprint from key column measures and compares it to the prior run's fingerprint. A changed fingerprint indicates the table has been updated; a matching fingerprint means the table is stale.
When to use: This test tracks whether a table has been refreshed since the last check. It builds update history that feeds into freshness monitoring. Unlike the Freshness monitor which evaluates timing against an expected window, Table Freshness simply detects whether any change occurred.