How to Migrate a Connection from Snowflake Native Tables to Lakehouse Tables

Use this tutorial to move a connection from a Snowflake destination that uses Native tables to another Snowflake destination that uses Lakehouse tables.

The migration uses the CONTINUE_WITH_DATA sync behavior of the Move a Connection REST API endpoint. Fivetran copies the existing data to the Lakehouse tables destination and preserves the connection's sync cursor, so you do not need to perform a historical re-sync.

Prerequisites

Before you begin, make sure you have the following:

• A scoped API key and secret

• A Snowflake destination that uses Native tables and contains the connection you want to move

• A Snowflake destination that uses Lakehouse tables and meets the following requirements:

  • Uses the same Host and Snowflake database as the destination that uses Native tables
  • Has an associated Managed Data Lake Service destination with a storage region that matches the Snowflake compute region to help avoid data egress costs

• The connection_id of the connection you want to move and the destination_group_id of the target Lakehouse destination

• A connection that does not use the Source naming convention

• A connection that meets the requirements in Move a Connection endpoint documentation, including the additional requirements for CONTINUE_WITH_DATA

• jq installed on the machine from which you run the API commands

Endpoints used in this workflow

• PATCH /v1/connections/{connectionId}
• POST /v1/connections/{connectionId}/move
• GET /v1/connections/{connectionId}/move/{jobId}

Instructions

Follow these steps to move the connection and verify that the migration completes successfully.

Set environment variables

Set the environment variables required for REST API authentication and the migration:

export FIVETRAN_API_KEY="<your_api_key>"
export FIVETRAN_API_SECRET="<your_api_secret>"
export FIVETRAN_BASE_URL="https://api.fivetran.com/v1"
export FIVETRAN_AUTH_HEADER=$(printf "%s" "$FIVETRAN_API_KEY:$FIVETRAN_API_SECRET" | base64 | tr -d '\n')

export CONNECTION_ID="<your_connection_id>"
export LAKEHOUSE_DESTINATION_GROUP_ID="<your_lakehouse_destination_group_id>"

Pause the connection

Pause the connection before moving it:

curl --silent --show-error \
  --request PATCH \
  --url "$FIVETRAN_BASE_URL/connections/$CONNECTION_ID" \
  --header "Accept: application/json" \
  --header "Authorization: Basic $FIVETRAN_AUTH_HEADER" \
  --header "Content-Type: application/json" \
  --data '{"paused": true}'

Move the connection

Use the Move a Connection endpoint with sync_behavior set to CONTINUE_WITH_DATA:

MOVE_RESPONSE=$(curl --silent --show-error \
  --request POST \
  --url "$FIVETRAN_BASE_URL/connections/$CONNECTION_ID/move" \
  --header "Accept: application/json" \
  --header "Authorization: Basic $FIVETRAN_AUTH_HEADER" \
  --header "Content-Type: application/json" \
  --data @- <<JSON
{
  "destination_group_id": "$LAKEHOUSE_DESTINATION_GROUP_ID",
  "sync_behavior": "CONTINUE_WITH_DATA"
}
JSON
)

echo "$MOVE_RESPONSE"
export MOVE_JOB_ID=$(echo "$MOVE_RESPONSE" | jq -r '.data.job_id')
echo "MOVE_JOB_ID=$MOVE_JOB_ID"

The response includes the following fields:

• data.job_id: The identifier of the asynchronous migration job. The command stores this value as MOVE_JOB_ID. Use it in the next step to monitor the migration.
• data.moved_at: Remains null while the CONTINUE_WITH_DATA migration runs asynchronously.

Monitor the migration job

Use the Retrieve Move Connection Job Status endpoint to check the migration status until the job finishes:

for i in {1..30}; do
  JOB_STATUS_RESPONSE=$(curl --silent --show-error \
    --request GET \
    --url "$FIVETRAN_BASE_URL/connections/$CONNECTION_ID/move/$MOVE_JOB_ID" \
    --header "Accept: application/json" \
    --header "Authorization: Basic $FIVETRAN_AUTH_HEADER")

  JOB_STATUS=$(echo "$JOB_STATUS_RESPONSE" | jq -r '.data.status')
  echo "Attempt $i: status=$JOB_STATUS"

  if [ "$JOB_STATUS" = "SUCCESS" ] || [ "$JOB_STATUS" = "FAILED" ]; then
    break
  fi

  sleep 30
done

The data.status field returns one of the following values:

• PENDING
• IN_PROGRESS
• SUCCESS
• FAILED

Resume the connection

After the migration job reaches SUCCESS, resume the connection to continue syncing data to the Lakehouse tables destination. Fivetran does not automatically resume the connection after the migration.

curl --silent --show-error \
  --request PATCH \
  --url "$FIVETRAN_BASE_URL/connections/$CONNECTION_ID" \
  --header "Accept: application/json" \
  --header "Authorization: Basic $FIVETRAN_AUTH_HEADER" \
  --header "Content-Type: application/json" \
  --data '{"paused": false}'

Verify the migration

Use the Retrieve Connection Details endpoint to verify that the connection's group_id matches the destination_group_id of the target Lakehouse tables destination:

curl --silent --show-error \
  --request GET \
  --url "$FIVETRAN_BASE_URL/connections/$CONNECTION_ID" \
  --header "Accept: application/json" \
  --header "Authorization: Basic $FIVETRAN_AUTH_HEADER"