Skip to main content

Create Flow

You can create flows to organize your datasets, recipes, imported reference objects that you use to generate your results. You can create flows from the Flows page and Import Data page.

To create an end-to-end data pipeline, in your flow must create at least one of the following objects:

  • Imported dataset: A reference to source data.

  • Recipe: A set of transformation steps to apply to your source data.

  • Output: A definition for how the output is to be generated, where it is to be located, and the output format.

Steps

  1. From the menubar, click Flows.

  2. In the Flows page, click Create > Create Flow. A new flow is created, with the name Untitled - X, where X is a number.

    Tip

    You can click the Untitled - X to enter a flow name and description.

  3. Imported Dataset:

    1. In Flow View, click Add Datasets.

    2. From the Add datasets to flow dialog, you can add existing datasets or import new datasets.

      1. Add datasets:

        • You can browse or search for the datasets and add them to your flow.

        • To add a dataset, click the checkbox next to it.

      1. Import datasets: Click import datasets and add datasets from your local desktop or from the connected datastores.

      2. For more information, see Import Basics.

    3. When finished, click Add.

  4. Recipe:

    1. In Flow View, right-click the imported dataset and select Add > Recipe. A new empty recipe is created for the dataset.

      Tip

      You should rename your recipe as soon as you create it.

    2. To edit a recipe, select the recipe and click Edit Recipe in the right panel. The Transformer page is loaded with the initial data sample. For more information, see Transform Basics.

    3. To add a new step to your recipe:

      1. Click Add New Step in the recipe panel.

      2. In the Search panel, you can search for the transformations and add the required transformations.

  5. Output:

    • In Flow View, right-click the recipe and select Add Output. An output is created on the flow canvas. For more information, see Create Outputs.

  6. Run a job: A job executes the steps of the recipe on the imported dataset to generate the results, as defined by the output.

    1. To run a job on the output, right-click the output and select Run.

      Tip

      You can also click Run from the right side panel fo the Jobs panel.

    2. In the Run Job page, specify the job that you wish to run. By default, an output object is pre-defined to generate a CSV output file. Click Run.

    3. See Run Jobs.

In this section: