Skip to main content

Create a Plan

Note

This feature may not be available in all product editions. For more information on available features, see Compare Editions.

A plan is a sequence of tasks that are executed manually or based on a schedule. Plans can be used to automate the execution of multiple related tasks in the Dataprep by Trifacta platform.

  • When a plan is triggered:

    • A snapshot of the objects in the plan is capture. This snapshot defines the set of tasks that are executed as part of a plan run.

      Note

      A snapshot does not capture theassets underlying the tasks. After a snapshot is taken, subsequent changes to the underlying assets could impact the outcome of the tasks when they are later executed during the plan run.

    • The set of tasks in the plan are triggered in the order listed in the plan.

    • All of the dependencies for any task are also executed.

    • If one task fails to execute, the other tasks are not executed.

  • For more information on plans, see Overview of Orchestration.

Before You Begin

Before you begin, please verify the following:

  • You have access to all of the assets that you wish to use in your plan.

  • For each flow in your plan:

    • All of the recipes whose results you wish to generate have output objects associated with them.

    • Each output object has at least one of the following that has been created for it:

      • file-based output

      • table-based output

        Note

        In a flow, all recipes that you wish to have executed by the corresponding task must have a defined output object. For each output object, you must create at least one write file or table settings definition. During plan runs, these objects are not validated, and missing outputs are ignored.

Create Plan

To begin, you must create a plan object.

Steps:

  1. From the left nav bar, click the Plans icon.

  2. The Plans page is displayed.

  3. In the Plans page, click Create. A new plan with the name Untitled - X is created, where X is a number.

  4. Click the Untitled - X to enter a plan name and description.

  5. Your plan is saved and displayed in Plan View.

In Plan View, you create the objects that are part of your plan. These include:

  • Plan Schedule: A schedule is composed of one or more triggers that determine when the plan is executed.

    • Trigger: Scheduling object that determines the conditions under which the plan is executed.

    • A schedule can contain one or more triggers.

  • Task: An action that is executed when triggered.

    • You can build a sequence of one or more tasks in your plan.

Create Task

Steps:

  1. Identify the tasks that you wish to execute. See "Task Types" below.

    Note

    You must have access to any assets that you wish to execute as tasks.

  2. Add a task.

    1. In Plan View, click the Plus icon at the bottom of your plan.

    2. Specify the task to execute.

  3. Repeat the previous step to add additional tasks as needed.

    Tip

    You can insert tasks between other tasks. Use the Plus icon between two plan objects.

  4. To test your plan, clickRun now. The plan is immediately queued for execution.

  5. Edit the plan and repeat the above steps until the plan is ready for production runs.

    Tip

    While a plan is in development, you may wish to disable its schedule, which prevents execution according to the schedule. You can still run test executions using the Run Now button.

  6. Create the schedule for the plan.

    1. In the context menu for the plan, select Schedule.

    2. Specify one or more triggers for the schedule. When a trigger occurs, the plan is queued for execution.

  7. When ready, the plan runs at the time scheduled in the trigger.

Task types

  • Flow task: For more information, see Plan View for Flow Tasks.

  • HTTP task: An HTTP task is a request sent using HTTP protocol to a target URL, which could be a REST API endpoint. For more information, see Create HTTP Task.

  • Slack task: A Slack task is a message submitted from the Trifacta Application to a specified Slack channel. For more information, see Create Slack Task.

  • Delete task: A Delete task deletes a specified set of files or folders from backend storage. For more information, see Create Delete Task.

Create Plan Branches

In some scenarios, you may need to branch plan execution steps based on the results of a task in the plan. For example, you may need to send separate messages using an HTTP task depending on whether a previous task succeeds or fails in execution. You can create branches in the plan graph by adding task execution rules and parallel nodes, which run based on the success and failure states of your plan runs.

To begin this simple example:

  1. Create your first task.

  2. Complete the following sections.

Add Task Execution Rule

Next, you create the first HTTP task that results from the above task and the execution rule that determines when it runs.

  • This task should run based on the successful execution of the first task.

  • A task execution rule is a condition that is tested after a task has run to determine if the task that is downstream of it is executed as a result. In this case, you create an On success rule.

Steps:

  1. Click the plus icon below the existing task node.

  2. Select the HTTP task type and enter information in the required fields. See Add HTTP Tasks above.

  3. Click the link connecting the created HTTP task node and its previous task node and select On success.

The HTTP task is executed only when the task has run successfully.

Add Parallel Task

Next, you can create the HTTP task that runs when the first task fails.

Steps:

  1. Click the plus icon below the first task node and select Add a parallel node. A parallel node is added to the plan graph. See Example below.

  2. Select the HTTP task type and enter information in the required fields. See Add HTTP Tasks.

  3. Click the link connecting the new HTTP task and its previous task node and select On failure.

The second HTTP task is executed only when the first task has failed to execute.

Tip

You can use parallel tasks to create separate paths through a plan when there are no dependencies between the paths.

Example - Success or Failure Tasks in a Plan

CreateAPlan-CreateBranchingPlan.png

Figure: Success and Failure tasks

When the first task complete successfully, the On success HTTP task sends a message.

When the task fails, the On failure HTTP task delivers a different message.

Apply Parameter Overrides

If your plan tasks include assets where which parameters have been defined, you can review and override these parameter values. Overrides are applied when the task is triggered as part of a plan run.

Steps:

  1. From the Plan View context menu, select Parameters.

  2. Review the names, sources, and current values for all of the parameters in your plan.

  3. To apply an override, click the Pencil icon and enter a new value. Click Save.

Subsequent runs of the plan use this new value as the override for the parameter.

Test Plan

After you have created the tasks of your plan, you can perform a test run of the plan.

Steps:

  1. To test, click Run now.

  2. The plan run is queued for execution.

Monitor Plan Runs

  1. in the upper-right corner of Plan View, click the Runs link.

  2. In the Plan Runs page, you can track the progress of your plan run.

    1. The most recently triggered plan run is displayed.

    2. If you have executed multiple runs, you can use the angle brackets next to the timestamp for the run.

  3. For tasks in progress, you can click the task to display information in the context panel.

  4. To see the details for the plan run, click the Outputs tab. Then, click Job details.

For more information, see Plan Runs Page.

Add Plan Schedule

You can add a schedule object to specify the triggers when the plan is to be executed.

Note

A plan's schedule cannot be executed until its schedule has been enabled. If a plan has a disabled schedule, you can still execute it via the Run Now button.

Steps:

  1. To begin, do one of the following, click Schedule from the Plan View context menu.

  2. In the right context panel, click Create schedule.

  3. In the Add Trigger panel, you can specify the triggers when the plan is executed. You can specify one or more triggers:

    CreateaPlan-AddTrigger.png

    Figure: Add trigger(s)

  4. For each trigger:

    1. Timezone: Specify the timezone that applies to the scheduled time. For more information on timezones, see Supported Time Zone Values.

    2. Frequency: You can specify the frequency of when the schedule is triggered.

      1. In each trigger, you can specify multiple On values (e.g. Same time on Sunday and Monday).

      2. As needed, you can specify the On value using a modified form of cron job syntax. For more information, see cron Schedule Syntax Reference.

  5. To add more triggers, click Add another trigger and specify it.

    1. To delete a trigger, click the X next to it.

  6. Parameter overrides:

    1. If the assets in your plan contain parameters, you can apply overrides to the parameter values.

    2. Overrides provided in this panel are applied only when the trigger is executed.

      Note

      Multiple values are ok for plan parameters, as long as the parameter values do not conflict. If you see a warning icon next to a set of multiple parameter values, then you must fix this conflict in the original asset, or the plan fails to execution.

    3. You can apply overrides through Plan View, too.

  7. To save your schedule, click Save.

  8. In the context panel, you can make changes to your schedule:

    1. After saving, the schedule is automatically enabled. To disable the schedule, use the slider bar.

      Note

      A plan cannot be executed if the schedule for it has been disabled.

    2. To make changes to the schedule and its triggers, click Edit.

In this section: