Check out this content on our new Help site.

Alteryx Engine and AMP: Main Differences

In the Alteryx AMP Engine article we have covered what is the Alteryx Engine and the new Alteryx Multi-threaded Processing (AMP). Now let’s go deeper into the main differences between the 2.

Data Processing Differences

The original Engine architecture allows for a single-threaded processing, where your data are processed record-by-record sequentially. On the other hand, the new AMP concept allows for massively multi-threaded processing. Records process in 4 MB packets for a faster run time, and in parallel, which can affect the output record order.

Input Differences

CSV files that contain a field with quoted newlines fail if you don't enable the additional option AMP only: quoted fields can have newlines in them.

Output Differences

Several tools might output records in a different order than the original engine when running a workflow with the AMP engine. These tools include...

  • Cross Tab
  • Data Cleansing (When you remove NULL rows.)
  • Join
  • Join Multiple
  • Multi-Row Formula
  • Poly Build
  • Running Total
  • Sort (an order with special characters differs from the original Engine).
  • Summarize (When you use Group By.)
  • Tile
  • Union
  • Unique

If your workflow requires records from the above tools to be in a specific order for downstream operations, the Engine Compatibility Modeoption is available to maintain the same order as the original Engine. This should be used after careful consideration of the specific workflow, mainly when migrating workflows that were created with the original Engine to run with AMP Engine.

Specific functionality or configuration that has not been converted to AMP reverts to the original engine tool to work. Therefore, workflows that contain both AMP-converted and non-converted tools run seamlessly with AMP.

If you have questions as to which tools might have been converted to AMP, go to Tool Use with AMP.

Read Performance

A YXDB file written with the AMP engine reads in faster than a YXDB written with the original Engine. The YXDB file written with the original Engine reads slower with AMP enabled. The formats are still compatible with each other. Use XLSX, CSV, YXDB, and SQLite file formats with AMP—they support multi-threaded, read-in data.

There is a performance cost to converting records and packaging between original Engine and AMP when reading in Zip files. This can cause larger Zip files to read significantly slower with AMP.

Tip

When opened in a text editor, a YXDB file written with AMP has "Alteryx e2 Database file" at the very beginning of the file content. A file written with the original Engine shows "Alteryx Database File" at the same place.

Write Performance

To improve the original Engine performance (make AMP write an YXDB file created with the original Engine), go to the Output Data - Configuration menu, where you have an option to create the version of YXDB file compatible with Designer 18.1 and older.

The Output tool behaves differently with records that contain SpatialObj data when saving a CSV file with the original and AMP engines. While AMP Engine writes SpatialObj data into the file when saving as CSV file, the original Engine doesn’t. Due to this fact, the original Engine produces a smaller file size and this difference causes the performance issue.

If needed, a workaround is to remove the spatial data from the records by using the Select tool, then both Engines will complete with similar durations.

Performance Profiling

Performance Profiling per Tool with AMP is available with Designer 2021.3 and newer.

Text Input Tool and AutoField

AMP addresses a historic issue where the size of the field may not be large enough when processed by a downstream tool. You don't need to add Select tools to change data types when the resulting data exceeds the length of the original data type. AMP creates the maximum size field for strings and integers so that subsequent operations have the necessary room to hold larger downstream values.

Throttle Tool

Although the Throttle tool was not fully converted to AMP, you can use it together with the Download tool (Throttle 1st.)

Fuzzy Match

Fuzzy Match may have different results between the original Engine and AMP. AMP records are matched using an alternative method. The order of match might be different and the output might be in reverse order as well.

There is a known performance issue where Fuzzy Match is less performant with AMP than the original Engine.

RegEx Tool

AMP uses Unicode and Perl encoding standards, where characters $+<=>^|~ don't qualify as punctuation. When you use the formula function REGEX_Replace or the REGEX tool to filter punctuation using the RegEx set [[:punct:]], with AMP you need to change the expression. For example: REGEX_REPLACE([_CurrentField_],'[[:punct:]]|[\$\+<=>\^`\|~]','')

Grouping Tools and Blocking Tools

The Join algorithm with the original Engine is based on a sort-merge join method, where the records always come in a sorted order. The new Join algorithm with AMP is based on a hash join method, so the record order comes out disordered.

If you need to have sorted order in join output, add the Sort tool after the Join or enable Engine Compatibility Mode in Workflow Configuration - Runtime, below the Use AMP Engine setting.

Iterative Macros

The difference between the original Engine and AMP can occur when a tool inside the macro reports an error. Being single-threaded, the original Engine stops if an error occurs in the macro. AMP works until the iterative output is empty or the maximum number of iterations occur.

You can encounter these situations, due to a higher number of iterations:

  • The number of errors (if any) can be higher with AMP.

  • The number of records can be higher with AMP.

  • The output schema can be different with AMP.

Formula Tool

ConvertFromCodePage and ConvertToCodePage functions in the Formula tool accept string as a parameter and return string as a result, so it is not possible to distinguish how the string is encoded. There is a difference in the output of the Formula tool with these functions used with the original Engine and AMP.

A different binary representation of the input data is caused by AMP's internal use of UTF-8 encoded strings. When data with a different encoding is imported, there is no way back to restore the original data.

The original Engine stores strings as Latin-1 or UTF-16 encoded strings that were used as a buffer and allow to convert data back correctly.

Formula Add-Ins

Formula Add-Ins are not yet supported with AMP. If you need to run a workflow that contains Formula Add-In functionality, run it using the original Engine.

Analytic App

Apps that use the Map tool to select from a spatial reference layer in an Analytic App should continue using original Engine.

Expect Equal

With the original Engine, Expect Equal remains a CReW macro. With AMP it runs as a native tool.

Parallel Branch Execution and Tool Run Order

Some workflows read from a file and then write back to it. This requires sequencing to ensure that the read is complete before the write can start. Similarly, a workflow that wants to write several sheets in one XLSX file needs to write the sheets one at a time. Alteryx Designer provides a Block Until Done (BUD) tool to help partition the work into phases that won’t get in each others way.

When working on a workflow with multiple branches (largely separate streams from inputs to outputs), place the BUD tool in the workflow branch with the lowest numbered Input tool ID. This ensures every subsequent branch waits to run until the previous branch is done and the tool works as expected.

Available Functionality

For more information regarding specific tool functionality, go to Tool Use with AMP.