Unity Catalog Setup Instructions

Follow our setup instructions to integrate Databricks Unity Catalog with your S3 Data Lake destination.

Setup instructions

Create a metastore and assign a workspace to it by following the instructions in Databricks' documentation.

Enable Unity Catalog for your workspace by following the instructions in Databricks' documentation.

Create your storage credentials by following the instructions in Databricks' documentation.
Log in to your Databricks workspace.
Go to Catalog > External Data.
Click External Locations.
Click Create location.
Select Manual and then click Next.
Enter the External location name.
In the Storage credential drop-down menu, select the credential you created.
In the URL field, enter the path to the S3 bucket you configured for your S3 Data Lake destination in Fivetran.
Click Create.

Create a notebook by following the instructions in Databricks’ documentation.

NOTE: You can create the notebook in any folder within your workspace.

To create an external table, execute the following SQL query from the notebook you created:

CREATE TABLE <catalog>.<schema>.<table>
USING delta
OPTIONS (
path 's3://<bucketName>/<path-to-table>'
)

In the SQL query, replace the following placeholder values with your actual values:

Placeholder Value	Actual Value
`<catalog>`	Name of the catalog where you want to create the external table.
`<schema>`	Name of your Unity Catalog schema.
`<table>`	Name you want to assign to your table in Unity Catalog.
`<bucketName>`	Name of the S3 bucket associated with your S3 Data Lake destination.
`<path-to-table>`	Path to the table within your S3 bucket.

NOTE: You can use the decode(unhex(<column_name>), 'UTF-8') clause in your query to fetch the decoded values in BINARY columns.