Skip to content

Get Started with Observability

DataKitchen's DataOps Observability provides end-to-end visibility across your data operations, so you can see how everything is going in real-time.

Observability collects metadata, metrics, and statuses from your data estate by ingesting events sent from your tools and data assets. The system processes the events, displays the details and results in the UI, and triggers actions based on the schedules, rules, expectations, and conditions you set—ensuring you can monitor essential situations.

Installation and Quick Start

Get started by installing the open-source version of Observability on your machine, and follow the quick start guide to explore its features and capabilities.

  1. Complete the prerequisites, installation, and demo setup steps described in Install on Windows or Install on Mac/Linux.
  2. Follow our Quick Start demo tutorial for a walk-through of the software.

Enterprise

For our enterprise Observability software, see Install Enterprise.

Use Observability on your data estate

After completing the quick start tour, set up Observability to monitor your tools to gain end-to-end visibility into your data operations.

  • Integrate your tools with Observability to ingest events from your own data estate.
  • Connect TestGen with Observability to add data quality results into the context of your data journeys.
  • Join our Slack community to ask questions and learn from other Observability users.

Terms to know

Here are some Observability terms to know when getting started.

  • Event: An event is anything of interest that happens in a component. Events are sent by your tools and received by the Event Ingestion API.
  • Component: Every component represents a type of tool or asset.
    • Run: A run is the execution of a batch pipeline component.
  • Journey: A journey is a collection of components and their dependencies, together responsible for creating a specific data analytic deliverable.
  • Instance: An instance is the runtime execution of a journey. That is, an instance is a snapshot of the events that happen for the components included in a journey.
  • Project: This is the space in Observability that separates and distinguishes work. In a customer account, projects might be set up by department or team. All data (events, components, journeys, instances, etc.) is unique to a project.

Tip

You'll often see Observability reference your data estate. This is a representation of your organization's data environment. The collection of all the data, tools, infrastructure, and assets that your data team is responsible for.