Loader Categories
On this page you find detailed information about the loader categories and which exact information is extracted by each of the loaders we support.
There are five types of the metadata loaders. Each type differs in the object hierarchy, structure of the objects and detail of the objects. To get more information about attributes for each loader family, select the corresponding tab below.
Database loaders are used to harvest metadata about database objects such as database server (RDMS), catalog, schema, table/view/procedure, column. An example can be an Oracle database with all available objects, descriptions, and relationships within the objects in the database.
Database Loaders
Connect supports database metadata loaders for these technologies:
Teradata, Snowflake, Apache Hive, IBM DB2, Exasol, Oracle, Databricks, Microsoft SQL Server, MySQL, SAP HANA, Amazon Athena, Amazon Redshift, Vertica, PostgreSQL, Google BigQuery, and Apache Impala.
File loaders are used to harvest metadata about file systems such as file bucket or folder with subfolder structure, individual files, sheets (when available for example on xlsx file), and columns on files you can parse for the structural information (xls(x), csv, yxdb, xml, twb, tde, avro).
For files with structure you can store also profiling information such as number of rows, not nulls, blanks, uniqueness, and average length.
File Loaders
Connect supports file metadata loaders for these technologies:
Amazon S3, Apache Hadoop, Microsoft Azure Data Lake, Databricks, Files loader (windows shared drives).
Visualytics are used to harvest metadata from reporting platforms such as report workbook, worksheet, datasource, and detailed columns.
Visualytics (Report) Loaders
Connect supports visualytics metadata loaders for these technologies: Tableau, Microsoft PowerBI, QlikView, Qlik Sense.
Workflow loader is used to harvest metadata from the Alteryx Gallery.
Workflow Loader
Connect supports only AYX Gallery Loader.
Other Loaders
This category describes loaders that don’t belong to any of the above-mentioned categories, such as:
Alteryx Promote loader is used to harvest metadata about Promote models published on a Promote server.
Microsoft Azure Data Catalog loader is used to exchange metadata from Azure Data Catalog about databases, files, and business terms.
Salesforce loader (without data profiling) is used to harvest metadata from the Salesforce application.
Overview of Available Objects for Each Loader Type
Loader Type | Available Objects |
Databases | db server, catalog, schema, table / view / procedure / column* |
Files | file bucket (AmazonS3) / folder, file, sheet*, column Possible data profiling (additional information about features and content of the files). |
Visualytics (reports) | server, site, project, workbook, worksheet, datasource, column Tableau: site, project, workbook, worksheet Microsoft PowerBI: workspace, workbook Insight QlikView: folder, Qlik data files (data profiling - .qvd, .qvx), report, worksheet Qlik Sense: stream, workbook, worksheet, report objects (names and types of charts, dimensions, measures) |
Workflows | workflow, input, output |
Others | Microsoft Azure Data Catalog: glossary, databases, files (without data profiling) Alteryx Promote loader: predictive model Salesforce loader: Salesforce tables |
*when applicable | |
How to Use and Configure Loaders
More information on how to use and configure the metadata loaders you can find on the following pages:
The pages Schedule Metadata Loaders and Load Metadata into Connect specify how to use the metadata loaders from Designer.
Now you can run metadata loaders directly in Connect by selecting Connections in the Admin Menu. You have the possibility to start the harvest immediately or to set up a regular harvest.
Go to Admin Menu > Connections to configure and run the metadata loaders.
For more info see: How To: Fast Deployment of Metadata Loaders in Connect.