Google BigQuery Output Tool Icon

Google BigQuery Output Tool

Version:
Current
Last modified: February 26, 2020

Use the Google BigQuery Output tool to write data from Designer to the tables in Google BigQuery. See Google BigQuery for information about known limitations. 

Alteryx Analytics Gallery

This tool is not automatically installed with Designer. To use this tool, download it from the Alteryx Analytics Gallery

Configure the Tool

Log in to Google BigQuery - Authentication Mode

You can log in to Google BigQuery with Service-to-Service or End user credentials.

Use Service-to-Service authentication over End user authentication whenever possible. Managing distinct keys for various applications authenticating for the same service account allows the revocation of access for a specific application without requiring the other applications to update their authentication credentials.

Contact your Google Cloud Administrator for help with your Google BigQuery credentials.

Service-to-Service Authentication

Service-to-Service authentication requires a Google Service Account File, also known as a service account JSON key file.

You can obtain a service account JSON key file from the Google Cloud Console, or you can create a new key for an existing service account. More information about Google BigQuery can be found on the Google Big Query Documentation site in Creating and Managing Service Account Keys.

  1. Under the Authentication drop-down, select Service-to-Service.
  2. Click the Select a file button to select your Google service account file.
  3. In the Open File pop-up window, select the JSON file that contains your Google service account key. Selecting the file enters your Google Cloud Service account key ID and Service account email.
  4.  Click the Select table button to display the BigQuery Table Output Configuration window, followed by selecting a BigQuery table. Select your project's drop-down menu. 

    If your key file is invalid, an error will display with the tool on your canvas, preventing you from running your workflow.

     

  5. Select your table under the project's drop-down menu. In Select table output mode, options include...
    1. Append Existing - Append to an existing table.
    2. Overwrite Table (Drop) - Drop the existing table, and create a new table with the same nameAppend to another existing table. When selecting a table, your existing table will be dropped then overwritten.

      Avoid frequent overrides of your tables, or data corruption will occur and your entire table will be dropped.

  6. In Insert batch size, select the size of your data that you want to batch. This number specifies the number of rows that will be written to the Google BigQuery table at each call to insert API.

    Google sets limits on the number of API requests per second, the number of rows that may be inserted per second and per request, the size of the rows to be inserted, and the size of HTTP requests.

End User Authentication

  1. In Authentication mode, select End user.
  2. Enter your Client ID and Client Secret.
  3. Select Log in.
  4. In Sign in with Google, enter your Google email and password then click Allow to approve the connector's permission to access Google BigQuery on your behalf.
    1. Enter your email address again for your Alteryx sign in, followed by selecting Next.
    2. Enter your password and select Sign in.
    3. Select the Advanced option, then select the Alteryx link.
  5. Select Next.

More information about limits and quotas can be found on the Google BigQuery Quotas and Limits Documentation site. You can also review the Streaming Data into BigQuery Documentation to understand how your data will be inserted into the Google BigQuery table.

Outputting data from your designer workflow to Google BigQuery streams new rows to the table in BigQuery. Data will be held in a temporary streaming buffer for up to 90 minutes while processes in Google BigQuery convert the row-based data to columnar-based storage. 

When the table that you want to overwrite still has data in streaming buffers, an error may occur.

Was This Helpful?

Running into problems or issues with your Alteryx product? Visit the Alteryx Community or contact support.