Connection Options

We support connecting to the following AWS-hosted sources using PrivateLink.

You must have an AWS endpoint service configured for your source before you set up a PrivateLink connection with Fivetran. AWS endpoint services only work with network load balancers (NLB), so you must create an NLB inside your VPC if you do not already have one. The NLB receives requests from Fivetran and routes it to your source.

The following graphic illustrates how Fivetran connects to the customer database using AWS Private Link:

Since endpoint service configurations are out of Fivetran’s control, we recommend that you contact your AWS representatives for help setting up PrivateLink. However, we do provide the following high-level instructions based on how customers typically configure their data sources:

1. In your VPC, create an NLB for your data source and configure it for each subnet (availability zone) in which the service should be available. For help, see the Create an NLB section.

   Skip this step if your data source is already running behind an NLB.

   If you chose to use security groups with your NLB, you must allow the following Fivetran's internal CIDR range on the security groups: 10.0.0.0/8.

2. Create a VPC endpoint service configuration and specify your NLB.

   Make sure that you are familiar with the endpoint service considerations and have met its prerequisites.

3. Safelist Fivetran’s AWS VPC Account ID (arn:aws:iam::834469178297:root) to allow access to your VPC endpoint service.

   To learn how to safelist the Fivetran account ID, see AWS’ endpoint service permission documentation.

4. Create a Fivetran support ticket and provide the service name (VPCe) (e.g., com.amazonaws.vpce.<region_id>.vpce-svc-xxxxxxxxxxxxxxxxx) along with the hostname of the service/source. Fivetran will then finish the setup on our side.

5. To activate the connection, accept the interface endpoint connection request from Fivetran. By default, connection requests must be manually accepted. However, you can configure the acceptance settings for your endpoint service so that any connection requests are automatically accepted.

6. As the PrivateLink setup is complete, you can now write your connector code to connect to the source using the hostname provided in Step 4. The code will run within Fivetran and securely access your source through the PrivateLink connection.

To set up PrivateLink for an on-premises source, please create a Fivetran support ticket requesting assistance. Our team will help you set up AWS Direct Connect to establish a private connection between your on-premises environment and an AWS VPC. Once the connection is in place, your Connector SDK code can use AWS PrivateLink to securely access your source through the VPC.

We support connecting to the following Azure-hosted sources using Private Link.

The following graphic illustrates how Azure Private Link establishes a connection between the Fivetran network and the customer network. Fivetran uses Private Endpoints to establish a Private Link with the Azure Database for MySQL and Snowflake.

To establish a Private Link connection, do the following:

1. Verify that your Azure-hosted source supports Private Endpoint in Microsoft’s Private Endpoint documentation.
2. Create a Fivetran support ticket and provide your Azure-hosted service’s fully-qualified resource ID, including the resource name and resource type of your Azure-hosted source.
3. Wait to receive Private Endpoint request details from Fivetran. We create a Private Endpoint using your resource ID, type, and subresource. We then initiate a Private Link connection request as part of the Private Endpoint setup and share the details of that request with you.
4. In the Azure Portal or CLI, verify and approve the Private Link connection request from Fivetran. Fivetran then finishes setting up Private Link for your Azure-hosted service on our side.
5. As the PrivateLink setup is complete, you can now write your connector code to connect to the source. The code will run within Fivetran and securely access your source through the PrivateLink connection.

How you set up Private Link for your Azure Virtual Machine-hosted or on-premises source depends on whether or not your source already has an Azure Private Link Service.

If your source is already configured with a Private Link Service, follow the setup instructions for Azure-hosted sources to complete your Fivetran setup.

If you do not have a Private Link Service yet, you will need to configure one before establishing a Private Link connection by following the steps below:

1. Inside the VNet, create a Standard Load Balancer with a NIC-based backend pool that has access to your data sources. Configure a health probe and a load balancer rule with the ports that your connections will use. Learn how in Azure's Create a load balancer guide.
2. Create a Private Link Service associated with the load balancer that you created in step 1. Learn how in Azure's Create a private link service guide.
3. Create one or more Virtual Machines and place them in the backend pool of the load balancer you created in step 1.
4. Enable IP forwarding on each of the backend Virtual Machine network interfaces. Learn how to enable IP forwarding in Azure's Enable IP forwarding documentation.
5. For security purposes, ensure that no public IP addresses are configured for the Virtual Machines. Private Link connections use only private IP addresses.
6. (Optional) Log in to each Load Balancer-backend Virtual Machine and run the script below. The script configures Network Address Translation (NAT), which forwards network packets from SLB-backend Virtual Machines to your data sources. The script below configures a single port forwarding, but feel free to add additional sets of iptables PREROUTING and POSTROUTING rules for additional ports:

#!/bin/bash

# This script configures Network Address Translation to forward incoming packets
# from the Azure Standard Load Balancer to IP-based destinations and route them
# back. In this script, "destination" means a data source server that Fivetran connects to.

# local port where Load Balancer sends traffic to
SOURCE_PORT=<local_port>
# destination server inside internal network
DESTINATION_IP=<destination_server_ip_address>
DESTINATION_PORT=<destination_server_port>

# enable IP forwarding on host
echo 1 > /proc/sys/net/ipv4/ip_forward

# clear existing iptables rules and chains
iptables -F
iptables -t nat -F
iptables -X

# change the packet recipient from local to destination socket (host & port)
iptables -t nat -A PREROUTING -p tcp --dport ${SOURCE_PORT} -j DNAT --to-destination ${DESTINATION_IP}:${DESTINATION_PORT}

# change the source IP address from the LB NAT IP address to the IP of this LB-backend host
iptables -t nat -A POSTROUTING -p tcp -d ${DESTINATION_IP} --dport ${DESTINATION_PORT} -j SNAT --to-source $(hostname -i)

For additional help, read Microsoft’s Private Link Service documentation or create a Fivetran support ticket.

We support connecting to any GCP-hosted resource as long as it's supported by Fivetran and exposed through a Private Service Connect producer.

In the following example, we publish a service that runs on a specific instance. However, there are several other ways to expose your service. To learn more, ask your administrators for help or read Google's Publish managed services using Private Service Connect documentation.

You'll need the following in Inputs for commands used in steps below:

• <NETWORK>- the VPC network in which the exposed resource exists
• <SUBNET>- the subnetwork where the exposed resource exists
• <ILB_SUBNET> - the subnetwork used for allocation of internal load balancers addresses (forwarding rules)
• <PSC_NAT_SUBNET> - the subnetwork used for allocation IPs for each customer endpoint address
• <REGION> - the region where the exposed resource exists
• <ZONE> - the zone where the exposed resource exists
• <VM_NAME> - the VM on which the exposed resource runs
• <VM_IP> - the private IP on which the exposed resource is available
• <RESOURCE_PORT> - the port on which the exposed resource is available in a VM
• <NETWORK_ENDPOINT_GROUP> - the network endpoint group
• <HEALTH_CHECKS> - the port health checks
• <BACKEND_SERVICES> - the backend services
• <FORWARDING_RULE> - the forwarding rule
• <SERVICE_ATTACHMENT> - the service attachment

1. Create a network endpoint group.

   gcloud compute network-endpoint-groups create --network <NETWORK> --subnet <SUBNET> \
    --network-endpoint-type gce-vm-ip --zone <ZONE> <NETWORK_ENDPOINT_GROUP>
   

2. Add an instance with the running resource as an endpoint to the network endpoint group.

   gcloud compute network-endpoint-groups update --zone <ZONE> <NETWORK_ENDPOINT_GROUP> \
    --add-endpoint='instance=<VM_NAME>
   

3. Create health checks to automatically enable and disable the instance. In this example, we use port checks.

   gcloud compute health-checks create tcp --region <REGION> --check-interval=60s 
    --port=<RESOURCE_PORT> <HEALTH_CHECKS>
   

4. Create backend services.

   gcloud compute backend-services create --region=<REGION> --health-checks=<HEALTH_CHECKS> \
       --health-checks-region=<REGION> --load-balancing-scheme=INTERNAL <BACKEND_SERVICES>
   

5. Assign the network endpoint group you created as a backend.

   gcloud compute backend-services add-backend <BACKEND_SERVICES> --network-endpoint-group=<NETWORK_ENDPOINT_GROUP> \
       --network-endpoint-group-zone=<ZONE>
   

6. (Optional) Allocate a subnetwork dedicated to forwarding rules.

   cloud compute networks subnets create --network <_NETWORK_> --region <REGION> 
       --range=10.0.X.0/24 <ILB_SUBNET>
   

7. Create a forwarding rule for the internal load balancer. It points to <BACKEND_SERVICES> and allocates the IP address from <ILB_SUBNET> in <NETWORK>.

   gcloud compute forwarding-rules create --backend-service=<BACKEND_SERVICES> --region <REGION> \
       --load-balancing-scheme=INTERNAL --ports=ALL --subnet=<ILB_SUBNET> --network=<NETWORK> <FORWARDING_RULE>
   

8. Allocate a NAT subnetwork dedicated to creating endpoints for each connected customer.

   cloud compute networks subnets create --network <NETWORK> --region <REGION> --range=10.0.X.0/24  \
    --purpose=PRIVATE_SERVICE_CONNECT <_PSC_NAT_SUBNET_>
   

9. Open traffic from the ILB and NAT networks to a VM or port using the corresponding ranges specified in previous steps.

   gcloud compute firewall-rules create --direction=INGRESS --priority=1000 --network=<_NETWORK_> --action=ALLOW \
       --rules=tcp:_RESOURCE_PORT_ --source-ranges=10.0.X.0/24 --destination-ranges=<_VM_IP_>/32 db-demo-allow-psc-nats
   
   gcloud compute firewall-rules create --direction=INGRESS --priority=1000 --network=<_NETWORK_> --action=ALLOW \
       --rules=tcp:_RESOURCE_PORT_ --source-ranges=10.0.X.0/24 --destination-ranges=<_VM_IP_>/32 db-demo-allow-ilb-nats
   

10. Create a service attachment and point it to the internal load balancer you created above (FORWARDING_RULE) with manual accepting mode.

    gcloud compute service-attachments create --producer-forwarding-rule=<FORWARDING_RULE> --connection-preference=ACCEPT_MANUAL \
        --region=<REGION> --description='Producer for my resource in region' --nat-subnets <_PSC_NAT_SUBNET_> <SERVICE_ATTACHMENT>
    

11. Create a Fivetran support ticket to set up a Private Service Connect link on the Fivetran side. Fivetran will provide a PSC_CONNECTION_ID, which allows you to identify which connections come from Fivetran before you approve them.

To connect your on-premises source to Fivetran, you must first use Dedicated Interconnect to connect a VPC to your on-premises source, then use Private Service Connect to connect that VPC to Fivetran.

1. Follow the instructions in Google's Dedicated Interconnect provisioning overview to set up Dedicated Interconnect and use it to connect a VPC to your on-premises source.

2. When setting up access for on-premises databases using private IP, you need an additional VM to act as a forwarding proxy. You can use your solution like HAProxy or iptables.

   If you use the following vm script, make sure to persist iptables rules and consider using instance groups instead of standalone VMs.

   #!/bin/bash
   
   # This script configures Network Address Translation to forward incoming packets
   # from the Load Balancer to IP-based destinations and route them
   # back. In this script, "destination" means a data source server that Fivetran connects to.
   
   # local port where the Load Balancer sends traffic to
   SOURCE_PORT=<local_port>
   # destination server inside internal network
   DESTINATION_IP=<destination_server_ip_address>
   DESTINATION_PORT=<destination_server_port>
   
   # enable IP forwarding on host
   echo 1 > /proc/sys/net/ipv4/ip_forward
   
   # clear existing iptables rules and chains
   iptables -F
   iptables -t nat -F
   iptables -X
   
   # change the packet recipient from local to destination socket (host & port)
   iptables -t nat -A PREROUTING -p tcp --dport ${SOURCE_PORT} -j DNAT --to-destination ${DESTINATION_IP}:${DESTINATION_PORT}
   
   # change the source IP address from the LB NAT IP address to the IP of this LB-backend host
   iptables -t nat -A POSTROUTING -p tcp -d ${DESTINATION_IP} --dport ${DESTINATION_PORT} -j SNAT --to-source $(hostname -i)
   

3. Once you have the VMs properly set up, follow the standard setup instructions for Google Cloud Private Service Connect.