Amazon RDS for MariaDB Setup Guide

Fivetran connects directly to your Amazon RDS for MariaDB database. This is the simplest method.

If you connect directly, you will create a rule in a security group that allows Fivetran access to your database instance.

Fivetran connects to a separate server in your network that provides an SSH tunnel to your database. You must connect through SSH if your database is in an inaccessible subnet.

If you connect using SSH, you will configure your tunnel server's security group to allow Fivetran access and configure your database's security to allow access from the tunnel.

Before you proceed to the next step, you must follow our SSH connection instructions. If you want Fivetran to tunnel SSH over TLS, follow Amazon's TLS setup instructions to enable TLS on your database.

AWS PrivateLink allows VPCs and AWS-hosted or on-premises services to communicate with one another without exposing traffic to the public internet. PrivateLink is the most secure connection method. Learn more in AWS’ PrivateLink documentation.

Follow our AWS PrivateLink setup guide to configure PrivateLink for your database.

Fivetran connects to your database through the Proxy Agent, providing secure communication between Fivetran processes and your database host. The Proxy Agent is installed in your network and creates an outbound network connection to the Fivetran-managed SaaS.

To learn more about the Proxy Agent, how to install it, and how to configure it, see our Proxy Agent documentation.

Retrieve external ID

1. Open your Fivetran connection setup form.

2. In the Authentication Method drop-down menu, select IAM.

3. Copy the External ID shown in the form. You will need it when creating an IAM role in AWS.

   The External ID is unique to your Fivetran account but remains the same even if you close and reopen the form.

Create IAM policy for Fivetran

1. In your AWS IAM console, go to Access management > Policies, then click Create policy.

2. In the Create policy window, switch to the JSON tab and paste the following policy:

   {
         "Version": "2012-10-17",
         "Statement": [
               {
                     "Effect": "Allow",
                     "Action": [
                         "rds-db:connect"
                     ],
                     "Resource": [
                         "arn:aws:rds-db:{region}:{account-id}:dbuser:{DbiResourceId}/{db-user-name}"
                     ]
               }
         ]
   }
   

3. Replace the placeholders:

   • {region} – your DB instance’s AWS region (for example, us-east-1)

   • {account-id} – your AWS account number

   • {DbiResourceId} – your RDS instance’s Resource ID (You can find it in the AWS RDS console under Configuration.)

   • {db-user-name} – the name of the database user for authentication using IAM. You can choose any name now — we will create the user during Step 6.

4. Click Next: Review Policy.

5. Name the policy (for example, Fivetran-RDS-Access).

6. Click Create policy.

Create the IAM role for Fivetran

1. In your AWS IAM console, go to Access management > Roles, then click Create role.

2. In the Create role window, select AWS Account as the trusted entity type.

3. Select Another AWS account, then in the Account ID field, enter Fivetran's AWS account ID: 834469178297.

4. In Options, select the Require external ID checkbox.

5. Enter the External ID you found in your connection setup form.

6. Click Next: Permissions.

7. Select the policy that you created earlier (for example, Fivetran-RDS-Access).

8. Click Next: Review.

9. Name the role (for example, Fivetran), then click Create role.

10. Select the Fivetran role you've just created.

11. Find the Role ARN and make a note of it. You will need it later.

1. In your Amazon RDS dashboard, select the MariaDB primary instance that you want to replicate.

   

2. Click Actions, then select Create Read Replica.

   

3. In the Settings section, enter your chosen instance ID.

   

4. In the DB instance size section, specify the instance class for the read replica. It does not need to be as large as your primary instance.

   

5. In the Connectivity section, set the Public access setting to Publicly accessible to ensure that the read replica is accessible from outside your VPC.

   

6. Click Create read replica.

7. The replica's status should now be creating. It will take a few minutes for the read replica to finish being created. The status will change to available when it is done.

   

1. In your Amazon RDS dashboard, click on the database that you want to connect to Fivetran.

   

2. In the Endpoint & port column, find the database's port number and make a note of it. You will need the port number to configure Fivetran.

   

3. In the Security column, click the link to the database instance's security group.

   

4. In the Security Group panel, click Actions, then select Edit inbound rules from the drop-down menu.

   

5. Click Add Rule. This creates a new Custom TCP Rule at the bottom of the list with a blank space for a Port Range and a Source IP address.

6. Fill in the new Custom TCP Rule.

   • In the Port Range field, enter your database's port number that you wrote down in Step 2 of this section. The port number will be 3306 for direct connections, unless you changed the default.
   • What you enter in the Custom IP field depends on whether you're connecting directly or using an SSH tunnel.
     • If you're connecting directly, enter Fivetran's IPs for your database's region.
     • If you're connecting using an SSH tunnel, enter {your-ssh-tunnel-server-ip-address}/32.
   • (Optional) Enter a brief description in the Description field.

   

7. Click Save rules.

If your database instance is not in a VPC, skip ahead to Step 6.

1. Return to the RDS Dashboard.

2. Click on your MariaDB database.

   

3. In the Networking column, click the link to the instance's VPC.

   

4. Select the VPC.

   

5. In the Summary tab, click the Network ACL link.

   

6. On the Network ACLs page, click the Network ACL ID.

   

You will see tabs for Inbound Rules and Outbound Rules. You must edit both.

Edit inbound rules

1. Go to the Inbound Rules tab.

   

2. If you have a default VPC that was automatically created by AWS, the settings already allow all incoming traffic. To verify that the settings allow incoming traffic, confirm that the Source value is 0.0.0.0/0 and that the ALLOW entry is listed above the DENY entry.

   

3. If your inbound rules don't include ALL - 0.0.0.0/0 - ALLOW entry, edit the rules to allow the Source to access the port number of your database instance. (The port will be 3306 for direct connections, unless you changed the default.) What you enter in the Source field depends on whether you're connecting directly or using an SSH tunnel.

   • If you're connecting directly, enter Fivetran's IPs for your database's region.
   • If you're connecting using an SSH tunnel, enter {your-ssh-tunnel-server-ip-address}/32.

For additional help, see Amazon's Network ACLs documentation.

Edit outbound rules

1. Go to the Outbound Rules tab.

   

2. If you have a default VPC that was automatically created by AWS, the settings already allow all outbound traffic. To verify that the settings allow outbound traffic, confirm that the Destination value is 0.0.0.0/0 and that the ALLOW entry is listed above the DENY entry.

   

3. If your outbound rules don't include an ALL - 0.0.0.0/0 - ALLOW entry, edit the rules to allow outbound traffic to all ports 1024-65535 for the following Destination(s):

   • If you're connecting directly, enter Fivetran's IPs for your database's region.
   • If you're connecting using an SSH tunnel, enter {your-ssh-tunnel-server-ip-address}/32.

Create a Fivetran database user

1. Open a connection to your primary database using your preferred SQL client (for example, MySQL Workbench or the mysql command in your operating system's terminal window).

2. Create a Fivetran user using one of the following commands, depending on your authentication method:

   • For password authentication, execute the following SQL command. Replace <username> and <password> with a username and password of your choice:

     CREATE USER '<username>'@'%' IDENTIFIED VIA mysql_native_password USING '<password>';
     

   • For IAM authentication, execute the following SQL command. Replace <username> with a username that matches the {db-user-name} from Step 2.

     CREATE USER '<username>'@'%' IDENTIFIED WITH AWSAuthenticationPlugin AS 'RDS';
     

Assign required permissions to the Fivetran user

Grant the following privileges to the Fivetran user you created in the previous step:

GRANT SELECT, REPLICATION CLIENT, REPLICATION SLAVE ON *.* TO '<username>'@'%';
GRANT SELECT ON mysql.rds_heartbeat2 TO '<username>'@'%';
GRANT SELECT ON mysql.rds_configuration TO '<username>'@'%';
GRANT EXECUTE ON PROCEDURE mysql.rds_kill TO '<username>'@'%';

Why these permissions are needed:

• Fivetran requires SELECT permissions for all columns in the tables you want to sync. If we don’t have full column access, we trigger a full re-sync for that table, which can significantly slow down your sync process. If you prefer not to sync certain columns, you can exclude them directly in the Fivetran dashboard instead of limiting SELECT access at the database level.

• Granting access to mysql.rds_heartbeat2 allows Fivetran to verify that the connector is correctly configured for your RDS database. Access to mysql.rds_configuration is also required to read sync interval settings and ensure proper incremental sync behavior.

• Granting EXECUTE ON PROCEDURE mysql.rds_kill is optional but recommended. This privilege allows Fivetran to terminate idle or orphaned connections that it creates but no longer needs. If you omit this grant, syncs can still succeed; however, you may occasionally need to manually close unused connections on your database.

Make sure these commands complete without any errors. If there are errors, you may lack sufficient privileges and should contact your database administrator.

Configure binary logging

Update your database's default RDS configuration to enable binary logging. We need binary logs to perform incremental updates.

Change binary logging format

1. Go to the AWS RDS Console and click Parameter groups.

   

2. Click Create parameter group.

   

3. On the Create parameter group page, enter the following information:

   • In the Parameter group family drop-down menu, choose the most current major version for your database. For example, if your database version is 10.4.13, your parameter group family value would be mariadb10.4.
   • In the Group name field, enter a name for the parameter group.
   • In the Description field, enter a brief description of the parameter group.

   

4. Click Create. You will be redirected to the Parameter groups page.

5. Click on the new parameter group.

   

6. In the Parameters section, search the binlog_format parameter, then click Modify.

   

7. In the Values field, type ROW, then click Continue.

   

8. Click Apply Changes.

Additional Settings (if using a read replica with history mode)

If you have connected Fivetran to a read replica and plan to run your connection in history mode, you may need to adjust several MariaDB replication parameters to avoid replication gaps and ensure correct binary log ordering.

1. In the RDS Console, view your read replica’s configuration.

2. Locate the slave_parallel_threads parameter.

   

   • If the slave_parallel_threads value is 0, no changes needed. Proceed to the Turn on automated backups section.

   • If the slave_parallel_threads value is not 0, you must set the slave_parallel_mode value to optimistic, conservative, aggressive, or minimal to use in-order parallel replication. Learn more about in-order parallel replication in MariaDB's parallel replication documentation.

     

     During in-order parallel replication, the transactions are executed from the relay logs on a replica in the same order as they happen on the primary database. It is very important to preserve the order of the events in the binary log, because Fivetran assumes that your binary log events are in sequential order during incremental updates.

Turn on automated backups

1. In the AWS RDS Console, click Databases in the left menu.

   

2. Select your MariaDB database, then click Modify.

   

3. In the Modify DB Instance screen, scroll down to find the Additional configurations section.

4. Set the DB Parameter Group value to the one you created.

   

5. Scroll down to Backup.

6. Set the Backup Retention Period to 1 day or more.

   

7. Click Continue.

8. Select Apply Immediately and click Modify DB Instance.

   

9. The Parameter Group setting in the database details should now show the name of your new parameter group. The Parameter Group status will say "applying" at first. Wait until the status changes to "pending-reboot."

   

10. For the changes to take effect, reboot your instance by clicking Actions > Reboot.

    

11. Click Confirm to reboot the instance.

    

12. Rebooting takes a few minutes. The configuration change is complete when the database's status changes to "available" and the parameter group status changes to "in-sync."

    

Set binary log retention period

To retain binary logs long enough for recovery or restart scenarios, Fivetran recommends setting the binary log retention period to seven days (168 hours). Run the following command on the Amazon RDS for MariaDB database that you want to connect to Fivetran:

CALL mysql.rds_set_configuration('binlog retention hours', 168);

Grant Fivetran access to binary log retention settings (optional)

During the connection setup process, Fivetran can check your binary log retention period and alert you if you need to set a longer retention period. To allow Fivetran to monitor your retention period:

GRANT SELECT ON mysql.rds_configuration to '<username>'@'%';

1. Open a connection to your primary database using your preferred SQL client (for example, MariaDB Client, DBeaver, MySQL Workbench or the mysql command-line tool).

2. Create a Fivetran user using one of the following commands, depending on your authentication method:

   • For password authentication, execute the following SQL command. Replace <username> and <password> with a username and password of your choice:

     CREATE USER '<username>'@'%' IDENTIFIED VIA mysql_native_password USING '<password>';
     

   • For IAM authentication, execute the following SQL command. Replace <username> with a username that matches the {db-user-name} from Step 2.

     CREATE USER '<username>'@'%' IDENTIFIED WITH AWSAuthenticationPlugin AS 'RDS';
     

3. Grant permissions to the Fivetran user created in the previous step.

   • Option 1 - Grant SELECT on all tables and columns:

     GRANT SELECT ON *.* TO '<username>'@'%';
     

   • Option 2 - Grant SELECT on specific tables or columns only:

     GRANT SELECT ON <your_database.your_table> TO '<username>'@'%';
     

     -- For column-level control
     GRANT SELECT (<column1>, <column2>) ON <your_database.your_table> TO '<username>'@'%';
     

   • Optional - Allow Fivetran to terminate its own stale sessions:

     GRANT EXECUTE ON PROCEDURE mysql.rds_kill TO '<username>'@'%';
     

4. Make sure these commands complete without any errors. If there are errors, you may lack sufficient privileges and should contact your database administrator.