Skip to content

Get Database Values for Testing

  • Use Case: A recipe includes nodes that migrate data from one database into another database. The data analyst wants to ensure that the data migrates properly without corruption.
  • Example Ingredient: In this example, an ingredient variation uses a single container node to query the SQL Server Analysis Service using credentials and source paths supplied as inflowing variables. It then outputs the sum of all values in a column as a variable that can be used in a comparison test. The ingredient may be used in other variations that use similar SQL datasets.
  • Example Recipe: In this example, the variation accesses the company's old Oracle database, sums the values in a specific column, then migrates the data to a SQL Server instance. It then sums the values in the same column and compares the result to the value obtained earlier. From there, the workflow performs several data analysis steps.
  • Ingredient Node Role: The ingredient node appears in the workflow after the data migration is complete. It calls the ingredient that queries the SQL Server, then it tests the output value by comparing it to the value stored previously. The query sequence is somewhat complex, so it is easier for this recipe builder to call a pre-defined ingredient to do the work.

()

An ingredient node calls a shared, single-node ingredient that is pre-configured to query a SQL Server.

Ingredient Configuration

A recipe builder on another team configured the SSAS_Query ingredient prior to your team building its oracle_to_sql_migration variation that calls this ingredient. See Declare Ingredients for more information.

  • Variation declared as an ingredient in the Ingredient Settings tab
  • Inflowing Variables (required or optional) and Outflowing Variables set
  • Configurations and tests for the node included in the ingredient

The ingredient's database connections are scripted with inflowing variables rather than set in the Connections tab.

()

A variation declared as an ingredient sets input fields and lists output values for use in testing.

()

This ingredient runs a series of Python scripts, uses inflowing variables as connection parameters, and produces output variables for tests.

()

A test configured within the ingredient ensures a successful run.

Ingredient Node Configuration

With the query ingredient pre-configured, the example variation can call it using an ingredient node. After adding an ingredient node to the variation graph, a user must supply information to run the ingredient.

  • Mapping information supplied in the Ingredient tab
  • Inflowing Variable values entered as needed
  • Outflowing Variables noted for test configuration
  • Tests built for the ingredient node that incorporate outflowing variables from the ingredient

()

Ingredient node configuration with a "mapping" to the ingredient.

()

Ingredient node configuration includes setting inflowing values or accepting the ingredient's defaults.

()

A test in the ingredient node uses an outflowing variable from the ingredient.