Skip to main content

Loader Categories

On this page you find detailed information about the loader categories and which exact information is extracted by each of the loaders we support.

There are five types of the metadata loaders. Each type differs in the object hierarchy, structure of the objects and detail of the objects. To get more information about attributes for each loader family, select the corresponding tab below.

Database loaders are used to harvest metadata about database objects such as database server (RDMS), catalog, schema, table/view/procedure, column. An example can be an Oracle database with all available objects, descriptions, and relationships within the objects in the database.

Database Loaders

Connect supports database metadata loader for these technologies:

Teradata, Snowflake, Apache Hive, IBM DB2, Exasol, Oracle, Databricks, Microsoft SQL Server, MySQL, SAP HANA, Amazon Athena, Amazon Redshift, Vertica, PostgreSQL, Google BigQuery, and Apache Impala.

File loaders are used to harvest metadata about file systems such as file bucket or folder with subfolder structure, individual files, sheets (when available for example on xlsx file), and columns on files you can parse for the structural information (xls(x), csv, yxdb, xml, twb, tde, avro).

For files with structure you can store also profiling information such as number of rows, not nulls, blanks, uniqueness, and average length.

File Loaders

Connect supports file metadata loader for the following technologies:

Amazon S3, Apache Hadoop, Microsoft Azure Data Lake, Databricks, Files loader (windows shared drives).

Visualytics are used to harvest metadata from reporting platforms such as report workbook, worksheet, datasource, and detailed columns.

Visualytics (Report) Loaders

Connect supports visualytics metadata loader for these technologies: Tableau, Microsoft PowerBI, QlikView, Qlik Sense.

Workflow loader is used to harvest metadata from the Alteryx Gallery.

Workflow Loader

Connect supports only AYX Gallery Loader.

Other Loaders

This category describes loaders that don’t belong to any of the above-mentioned categories, such as:

  • Alteryx Promote loader is used to harvest metadata about Promote models published on a Promote server.

  • Microsoft Azure Data Catalog loader is used to exchange metadata from Azure Data Catalog about databases, files, and business terms.

  • Salesforce loader (without data profiling) is used to harvest metadata from the Salesforce application.

Overview of Available Objects for Each Loader Type

Loader Type
Available Objects

Databases

db server, catalog, schema, table / view / procedure / column*

Files

file bucket (AmazonS3) / folder, file, sheet*, column

Possible data profiling (additional information about features

and content of the files).

Visualytics (reports)

server, site, project, workbook, worksheet, datasource, column

Tableau: site, project, workbook, worksheet

Microsoft PowerBI: workspace, workbook Insight

QlikView: folder, Qlik data files (data profiling - .qvd, .qvx), report, worksheet

Qlik Sense: stream, workbook, worksheet, report objects (names and types of charts, dimensions, measures)

Workflows

workflow, input, output

Others

Microsoft Azure Data Catalog: glossary, databases, files (without data profiling)

Alteryx Promote loader: predictive model

Salesforce loader: Salesforce tables

*when applicable

How to Use and Configure Loaders

More information on how to use and configure the metadata loaders you can find on the following pages:

The pages Schedule Metadata Loaders and Load Metadata into Connect specify how to use the metadata loaders from Designer.

Now you can run metadata loaders directly in Connect by selecting Connections in the Admin Menu. You have the possibility to start the harvest immediately or to set up a regular harvest.

Go to Admin Menu > Connections to configure and run the metadata loaders.

For more info see: How To: Fast Deployment of Metadata Loaders in Connect.