Skip to main content

Download Tool Icon Download Tool

Use Download to retrieve data from a URL to use in downstream processing or to save to a file. The Download tool can also download or upload data via FTP and SFTP.

Non-FIPS Designer releases starting with 2022.3 have Peer Validation enabled by default. Go to Peer Validation Allow List to learn how to specify a list of problematic hosts in a flat file for which peer validation should be skipped.

Configure the Tool

Basic Tab

Use the Basic tab to set the mandatory controls for the Download tool.

  • Select the Use Data Connection Manager (DCM) checkbox to set up the connection using DCM. For more information, go to DCM .

    If the DCM Connection Method requires Username and Password, these credentials are used for basic authentication.

  • By selecting the Resolve on Payload checkbox, you can use DCM-specific aliases for Headers and Payload settings. The aliases can be used for both Name and Value while only Compose Query String/Body option is supported for Payload. Once you run the workflow, these aliases are automatically replaced by corresponding DCM values. Supported aliases:

    • {dcm:userName}

    • {dcm:password}

    • {dcm:host}

  1. URL Field: Specify the field from the incoming data stream that contains the URL (Uniform Resource Locator - Internet resource) to pull data from (or upload data to). This could simply be a Text Input tool where the URL is specified as a field value.

    When using the Download tool with DCM, the URL that data will be pulled from is split into 2 parts. The base of the URL is fetched from the DCM Data Source definition while the latter part (that is appended to the base URL) is fetched from the incoming data stream.

  2. Encode URL Text: When checked, the URL is encoded so unsafe ASCII characters are converted into a format that can be transmitted over the internet. An example of this would be the substitution of %20 for a space.

  3. Output: Specify how the returned data is formatted. The data can be returned in a data field or output to a file.

    • To a Field: Downloaded content is returned in the data stream as a data field. The downloaded contents will be in a single field called "DownloadData." You might have to parse this data using downstream tools like the Text To Columns tool, RegEx tool, or Formula tool.

      • String: Data is returned as a new, wide, string-type field. A wide string supports Unicode® characters. Select one of the Code Pages to use for encoding. Default is UTF-8. For the most consistent results, applications should use Unicode, like UTF-8 (code page 65001) or UTF-16, instead of a specific code page.

      • Blob: Data is returned as a new blob-type field. Blob is also known as Binary Large OBjects. Image files are usually stored in this format. To use the image, configure an Image tool downstream and specify Image or Blob field.

    • To a File

      • Temporary File: Data is output to a temporary file located in the user's temporary directory. For more on temp file handling in Designer, go to Alteryx and Temporary Files.

      • Filename from a Field: Data is output to a specific file where the file specification is in an incoming field. Use the dropdown to select the field that contains the file name to output to.

Headers Tab

Use the Headers tab to modify the HTTP headers sent with the web request.

  1. Add from the following constant values: Allows for adding fixed header values.

    To add the header "Accept: text/plain", enter "Accept" in name and "text/plain" in value.

  2. And values from these fields: Takes values from the record data and creates header values.

    For example to add the header "Expect: 100-continue", select a field from the input data named "Expect" that contains "100-continue."

Payload Tab

Use the Payload tab to set the HTTP Action you want to perform and optionally build the Query String or Body for the web request.

  1. HTTP Action: Select the HTTP Action for the web request:

    • GET (or FTP): Perform a GET request or download a file from an FTP or SFTP site. You don't have to enter any other options on this tab to download the data from your selected URL.

    • POST: Performs a POST request to the selected URL. Typically when using this option you specify a POST body using the below options.

    • PUT: Performs a PUT request. This option only allows you to take the query body from a blob field via the Blob Input Tool. Typically used to upload a file to the remote server.

    • DELETE: Performs a DELETE request. Typically used to ask the remote server to delete a specified resource.

    • HEAD: Performs a HEAD request. This asks the server to return the header data, but not the body data.

    • Custom: This allows you to enter a custom verb in a text box. Everything else about this request behaves the same as a POST. The POST verb is replaced by the custom verb just before the request is made.

      Verb Support

      These options only work if the remote server that you are sending the requests to supports a particular verb. Check the API documentation of the URL you are using to see which requests are supported.

  2. Choose from these Query String/Body options:

    • Compose Query String/Body:

      1. From the Following Constant Values: Adds constant name-value pairs to the query string/body

      2. And values from these fields: Takes name-value pairs from the incoming data record. The field name is the name used in the Query String/Body

      3. Content-Type: Controls how the name-value pairs are encoded.

        • Application/x-www-form-urlencoded: For example, Name1=Value1&Name2=Value2&Name3=Value3. Unsafe ASCII characters are automatically encoded, so make sure your data is not already encoded.

        • Multipart/form-data: Only available when using HTTP Actions POST and Custom.

    • Take Query String/Body from Field: Select the field in the input data which contains the Query String/Body data. Blob fields can be selected when using HTTP Actions POST, PUT, and Custom.

    • Use Following for Query String/Body: Text box allowing you to manually type in the Query String or Body contents.

    • All text data is UTF-8 encoded before being sent to the remote web server.

Connections Tab

Use the Connection tab to set up connection credentials.

  • Username (Optional): Enter the user name if required by the URL specification above. This is an optional field.

  • Password (Optional): Enter the password if required by the URL specification above. This is an optional field.

The credentials above are used for basic authentication.

  • Maximum Connections: Specify the maximum number of simultaneous transfers for the Download tool to perform. Transfers are only done in parallel when there are multiple input records sent to the Download tool. Multiple Download tools operate independently but don't typically function at the same time. For new Download tools added to a workflow, the default number of connections is 2 and the maximum number of connections is 32.

    Increasing the number of connections might reduce the total time taken to complete all transfers, but use caution not to set the number too high as it could overload the server being used. It is possible the server could quit responding, report errors, or even refuse connections if it believes you are misusing it. This is particularly important when accessing a public site that is not under your control. Most web browsers will do as many as 6 simultaneous transfers, but these would typically be relatively small transfers as a part of a web page. For a server inside your own business where you have more control over how it is configured, using a higher number of connections might be okay.

    Additionally, because the Download tool sends records downstream as transfers complete, it might result in a change in the order of records as they pass through the tool. If the order matters, make sure to sort the results or limit the number of connections to 1. Finally, note that empty URLs are processed ahead of those that require an actual transfer.

  • Timeout (seconds): Specify the number of seconds to wait before reporting a timeout due to an unresponsive connection. Select a number from 0 (never timeout) to 10,000.

  • Throttle Records: (unchecked by default) Check to limit the number of records that pass through the tool in a given minute. This is useful for specifying the number of requests sent per minute when there are limits to how many records can be processed.

  • Records Per Minute: Check the Throttle Records checkbox directly above to customize the number of records that you want to pass through the tool per minute.

    • The default value is 60.

    • The acceptable range is between 0 and 60,001 (not inclusive).

Other Considerations

To avoid connection delays when using the Download tool, ensure that the "Automatically detect settings" option is deselected within your account’s internet properties (Control Panel > Internet Options > Connections > LAN Settings).

In this section: