Pending or Missing Runs

One of the following run statuses appears in Observability to alert you:

Pending

Missing

And one of the corresponding instance alerts appears in Observability:

run missed

started late

Instance ended; component progress incomplete

Visual examples in Observability

Details

When you create an expected schedule for when batch pipeline component runs should start, the system expects events to be received during that time.

• If no events are received after the grace period passes, the system creates a placeholder run with the Pending status.
• If no events are received by the next expected start, the system changes the placeholder run status to Missing.
• The instances that should include the batch pipeline runs are assigned alerts that correspond to the status.

Potential causes

The reasons why a run does not meet your expectations can be unique to the components in your data estate. As an example, your external tools may be experiencing slowness, a work schedule may have stopped, or a particular production pipeline was not started. Regardless of the cause, Observability provides end-to-end visibility across your data operations, so that you know when these mistakes occur and you can find a solution.

Solution

When you set an expectation for runs to occur in your batch pipeline, and the outcome is not as expected, Observability uses the run status (Pending or Missing) and alerts to inform you.

Before you analyze a batch pipeline in your data estate, use Observability to get more details. Try these:

• Select Events from the menu > Batch Runs tab > click the Run Key of the affected run. Verify the data in the run details.
• Select Instances from the menu > click the ID of the instance that had the affected run. Analyze the data in the instance details.

With this information, you can dig deeper into the behavior of the components in your data estate and determine how batch pipelines may be affecting the efficiency and progress of your work.