Observability Integration Agents¶

For complex data estates with multiple components and assets, an efficient way to integrate with DataOps Observability is to use agents. Agents can scrape events from event streams, log files, or tool APIs and send the events to Observability for you.

Note

For more customized integrations, DataKitchen provides a client SDK to instrument your tools to publish events directly to the Event Ingestion API. See Publish Events to the API for technical details.

Observability agent overview¶

DataKitchen provides ready-made agents so you can monitor components in your data estate without making modifications to your existing setup. Each integration agent will actively retrieve or listen to events from a specific tool, then forward the events to the Event Ingestion API.

Deploy agents¶

Once configured, agents must be deployed in your data estate so events can be sent to Observability. All Observability agents run as Docker containers and can be deployed anywhere in your data estate that supports Docker.

For more information and an example, see Deploy Observability Agents.

Available agents¶

DataKitchen provides agents for the following tools.

Tip

Don't see the connection you need? We can build it for you. Submit a request.

Agent development¶

At DataKitchen, we believe in continuous integration and continuous delivery. The agents that we provide are categorized by their development lifecycle.

Production: agents developed and maintained to high robustness standards with new features.
Beta: agents undergoing testing and improvements. If changes to your existing setup are required, announcements will be made in the form of release notes.

Stay up-to-date with agent releases in the Agent Changelog.