Alteryx and Temporary Files
What makes Alteryx such an efficient tool is how it generates and accesses temporary files to process many different data types. It is very important to ensure there is enough temp space to handle these files.
Temporary files are saved to the user's specified temporary directory. This directory can be specified via the System Settings, User Settings, or Workflow Configuration. A sub-folder is created for each new Alteryx process under the user's temp directory and the name of the sub-folder is tied to the engine process ID to ensure it is unique. This guarantees that any files generated by a particular Alteryx process are tied to that process alone and cannot be unexpectedly overwritten by another process.
By default, the temporary directory is: C:\ProgramData\Alteryx\Engine.
When a workflow or Analytic App is run, a new folder is created: C:\ProgramData\Alteryx\Engine\Engine_7336_31d79603f1024c3f9706faf2d5fc8ef8_. All temp files for that particular workflow are saved in this folder. At the end of the run, the files and directory are deleted.
Write to the Temporary Directory
To write files to a temporary location, use this shortcut to create a file path: %temp%filename.ext.
Specifying %temp% works on any machine and writes the file to the user's specified temporary directory in the sub-folder tied to the process ID.
Access the Temporary Directory
The workflow constant Engine.TempFilePath is available in every workflow and references the full path to the user's temporary directory, including the Process ID (unknown to the user at runtime.) Workflow constants are available in tools that use the expression box and can be used and parsed (if necessary) to construct one's own file path to ensure files can be written and read from a predictable, universal location. Go to Expression Editor content for more information.
Alteryx reads records in their entirety. Some tools need to read the contents in more than one pass, such as any tool that sorts the records prior to executing the tool process. Temp files are created so that the second pass can happen faster. Some tools that generate temp files include: Block Until Done tool, Find Nearest tool, Join tool, Join Multiple tool, Make Group tool, Poly-Build tool, Sample tool, Sort tool, Spatial Match tool, Summarize tool, Tile tool, and Unique tool.
- A temp file is created and written to the user-specified temp file location. Even though the file is compressed, it does contain the entirety of what was passed in the connection stream to one of the tools listed above. It is named according to the process that created it.
- Alteryx_[Process ID]_[GUI ID]_.ext
- These files usually have an Alteryx Database extension (*.yxdb.)
- As temp files are no longer necessary in the process, they clear themselves out. The lifespan of individual temp files differs, but in most cases, they are deleted as soon as that tool is done outputting all its records to downstream tools.
- Some temp files will not clear out until after the workflow is closed, such as temp files for Browse Data (*.yxdb files) and Browse Map (*.png files.)
- Orphaned” files (temp files remaining after a crash as in the case where temp space runs out) are automatically cleared out the next time the Alteryx Engine runs.