Skip to main content

Load Metadata from Google BigQuery

Use the BigQuery Loader to upload metadata to Alteryx Connect for specified BigQuery projects.

Review Loader Requirements

The Alteryx Connect Loaders must be installed on the machine where Alteryx Server is installed. To install the loaders:

  1. Download the loaders installer. For compatibility, the Loaders installer version must match the Alteryx Connect version.

  2. To approve the changes during the installation process, run the installer as an administrator. For more information see the Install Connect page.

For lineage requirements, see Maintain Data Lineage.

Review Google BigQuery Requirements

  • The Google Service Account Key File needs to be obtained to establish the connection to BigQuery.

  • To learn about service accounts, navigate to the Service accounts page.

  • To learn how to access all BigQuery projects you intend to load with the metadata loader, go to the Granting roles to service accounts page.

  • The service account needs to be granted with the bigquery.metadataViewer permission, or wider permissions. To learn how to grant the needed permissions, go to the Predefined roles and permissions page.

Open the Loader

  1. Open Alteryx Designer.

  2. Select Help > Sample Workflows > Alteryx Connect Metadata Loaders > BigQuery Loader > BigQuery Loader.

Run the App

  1. Select Run as Analytic App

    Run as analytic app button

    .

  2. In the BigQuery Server tab, select an application for the loader to use and enter the following::

    • Key path: Path to your Service Account Key file.

    • Projects to load: The comma-delimited list of BigQuery projects you want to load to Alteryx Connect. Leave empty to load all available projects.

    • Datasets to load: The comma-delimited list of BigQuery datasets you want to load to Alteryx Connect. The expected format for each dataset is <PROJECT_ID>.<DATASET_ID> where PROJECT_ID and DATASET_ID are names of projects and datasets as displayed in the BigQuery console. Leave empty to load all available datasets.

  3. In the Alteryx Connect tab, type the URL for your Alteryx Connect instance and your credentials:

    https://<yourcompany>.alteryxconnect.com

  4. Before running the app, select the floppy disk icon and save the app configuration to the following directory:

    C:\ProgramData\Alteryx\AlteryxConnect\BigQuery_Loader\

    Schedule Metadata Loaders

    Use the .yxwv app values file for scheduling the loader to run in the Gallery. See Schedule Metadata Loaders.

  5. Select Finish to run the app.

    • If the run succeeds, "Success" appears in the App Results window.

    • If the run fails, a "There were Errors" message appears.

    Depending on how much metadata is being imported, the app can take a long time to run.

How Filters Work

When you set both “Projects to load” and “Datasets to load” filter values, the loader loads only those datasets that belong to listed projects and vice versa.

For example:

  1. Projects to load = my-project, Datasets to load = <empty>.

    1. All datasets from my-project are loaded.

  2. Projects to load = <empty>, Datasets to load = my-project.my-dataset, my-other-project.my-other-dataset.

    1. Two listed datasets are loaded.

  3. Projects to load = my-project, Datasets to load = my-project.my-dataset, my-other-project.my-other-dataset.

    1. Only the dataset my-project.my-dataset is loaded. The dataset my-other-project.my-other-dataset is excluded from the load because my-other-project is not listed in Projects to load.

View the Metadata

  1. Go to the Alteryx Connect URL and log in.

  2. From the main menu, select Data Sources.

  3. Select a folder to view its contents.

In this section: