Skip to content

Components

Components represent the resources, engines, and tools that you use in your day-to-day work to deliver data analytic assets. Components can include batch pipeline runs executed by orchestrators, streaming pipelines in event-driven systems, datasets like database tables and files, and storage or computing infrastructure.

Every event received—or created—by the system is associated with a specific component.

Note

Component creation: Observability automatically creates a new component any time the system receives an event whose <component-type>_key does not match any existing component keys in the project. For more details, see Create Components.

Component types

Observability recognizes the following component types:

Component type Definition Example
  Batch pipeline A batch, or finite, tool for data engineering. An Apache Airflow DAG.
  Streaming pipeline Similar to a batch pipeline, but not finite. An event-based process enabled by Apache Kafka.
Dataset A specific collection of data. An Amazon Redshift table or a folder in an Amazon S3 bucket.
  Server Storage or computing infrastructure. An Amazon EC2 server running Airflow jobs.

View the Components page

Observability data is always associated with a specific project. To see a list of components:

  1. Use the drop-down in the menu to select a Project.
  2. Select Components from the menu.

The page displays the list of components alphabetically by name. Use the search and filters to narrow your results.

Component details

The information that displays for each component can include the following details:

  • The component name.
  • The key, an identifier—unique within the project—that associates incoming events with the correct component.
  • Who created the component. For example, a specific user or the system.
  • When the component was created.
  • The component type.
  • The external tool the component represents, if provided in the details.

Select a component to view its description and related journeys, and to manage the component details and expectations.

Tip

Select a component, then click View Events to open the Events page in a new tab, pre-filtered with the events received for that component.