Why Does Fivetran Sometimes Run MERGE Statements Instead of COPY Statements During an Incremental Sync?

Question

Why does Fivetran sometimes run MERGE statements instead of COPY statements during an incremental sync?

Environment

Destination: BigQuery

Answer

During each incremental sync, we use the source's primary keys to identify common data between records extracted from the source and those present in the destination. If the source doesn't have primary keys, we use the _fivetran_id value instead.

We run MERGE statements when we identify common data and COPY statements when we don't.