DocumentDB Setup Guide

Follow our setup guide to connect Amazon DocumentDB to Fivetran.

Prerequisites

To connect your DocumentDB to Fivetran, you need:

• DocumentDB version 4.0 or higher
• Your database primary instance's IP (e.g., 1.2.3.4) or domain (your.server.com)
• Your database's port (usually 27017)
• An SSH server

Setup instructions

Connect to DocumentDB

Create user

Create a database user for Fivetran using the DocumentDB Mongo shell.

1. Open the Mongo shell.

2. Connect to your primary node as an admin user.

3. Go to the admin database.

4. Execute the following command to create a user for Fivetran. Replace <username> and <password> with a username and password of your choice.

      use admin
      db.createUser({
        user: "<username>",
        pwd: "<password>",
        roles: [ "readAnyDatabase" ]
      })
   

Enable change streams

Fivetran uses change streams to perform incremental updates. You must explicitly enable change streams for all of the collections that you want Fivetran to sync.

Follow Amazon DocumentDB's Enabling Change Streams instructions to enable change streams.

Set change stream log retention duration

Set the change stream log retention duration so that it can retain at least 48 hours' worth of changes. We recommend increasing the size to retain seven days' worth of data.

Follow Amazon DocumentDB's Modifying the Change Stream Log Retention Duration instructions to adjust your change stream log retention duration.

Finish Fivetran configuration

1. In your connection setup form, enter a destination schema prefix. This prefix applies to each replicated schema and cannot be changed once your connection is created.

2. Choose your connection method. If you selected Connect via an SSH tunnel:

   • Copy the Public Key and add it to the authorized_keys file on your SSH server when configuring the SSH tunnel.

   • Enter the following SSH connection details:

     • SSH Host - Enter either an IP address (for example, 1.2.3.4) or a domain name (for example, your.server.com). Do not use a load balancer's IP address or hostname.
     • SSH Port - The port number for your SSH server.
     • SSH User - The username for the SSH connection.

   • If TLS is enabled on your DocumentDB cluster, ensure that the Require TLS through Tunnel toggle is turned ON.

3. (Not applicable to self-service private networking) In the Host and ports field, click + Add and enter the hostname and port of your primary node:

   • Use the format hostname:port or IP:port (for example, server.example.com:27017 or 1.2.3.4:27017).

   • (Optional) Click + Add again to provide the hostname and port for one or more replica nodes. Fivetran may use replicas to improve the initial sync speed and balance resource usage.

     Do not include any path components after the top-level domain.

4. (Only applicable to self-service private networking) From the PrivateLink connections drop-down menu, select your existing PrivateLink connection.

   • If you want to create a new PrivateLink connection, click + Configure a new PrivateLink connection and follow the setup instructions for self-service Fivetran accounts to configure the connection in your AWS account.
   • If TLS is enabled on your DocumentDB cluster, ensure that the Require TLS when using PrivateLink toggle is turned ON.

5. (Only applicable to self-service private networking) In the Port field, enter the port that matches your load balancer’s listener port.

   The port will be automatically appended to your PrivateLink endpoint URL.

6. Enter the Fivetran-specific User that you created in Create user step.

7. Enter the Password for the Fivetran-specific user.

8. Select your Pack mode.

9. Click Save & Test. Fivetran tests and validates our connection to your DocumentDB primary instance. Upon successful completion of the setup tests, you can sync your data using Fivetran.

Setup tests

Fivetran performs the following tests to ensure that we can connect to your DocumentDB primary and replica instances and that it is properly configured:

• The Connecting to SSH Tunnel Test validates the SSH tunnel details you provided in the setup form. It then checks that we can connect to your database using the SSH Tunnel.
• The Connecting to Host Test validates the database credentials you provided in the setup form. It then verifies that the database host is not private and checks that we can connect to the host.
• The Validating Certificate Test generates a pop-up window where you must choose which certificate you want Fivetran to use. It then validates that certificate and checks that we can connect to your database using TLS.
• The Connecting to Database Test connects to your all the database instances provided and verifies that your database is at least version 4.0. It then checks that we can access the schemas in your database.
• The Connecting to Database Test also checks if one of the instances provided is of type 'primary' by verifying that the change stream feature is enabled for at least one collection.