Snowflake Internal vs. External Staging

Question

Does Fivetran use internal or external staging for my Snowflake destination?

Environment

Destination: Snowflake

Answer

By default, Fivetran stages data using a Snowflake internal stage. If we encounter memory constraints or connectivity issues, Fivetran automatically falls back to external staging to ensure uninterrupted data loading.

Fivetran manages the staging process regardless of which staging method is used. We encrypt all staged data, generate temporary credentials for data loading, and automatically delete the staged files after the data is successfully loaded into Snowflake.

Context

In Snowflake's Overview of data loading documentation, they note that bulk load is the most efficient way to load large volumes of data. For the insert-only initial sync, Fivetran leverages staging to load data from files in the staging area directly into the target tables using COPY commands.

For updates following the initial sync, we process the changes using controlled routines. We use staging to bulk load incremental changes into staging tables, followed by MERGE statements to apply the changes to the target tables. Since the data already resides in Snowflake, the merges are very efficient, allowing for high throughput.