SingleStore Setup Guide Partner-Built Private Preview

Follow our setup guide to connect SingleStore to Fivetran.

NOTE: This connector is partner-built. For any questions related to SingleStore connector and its documentation, contact SingleStore by raising an issue in the SingleStore Fivetran Connector GitHub repository.

Prerequisites

To connect your SingleStore database to Fivetran, you need:

• A SingleStore instance of version 8.7.16 or higher. Refer to SingleStore's Creating and Using Workspaces documentation for instructions on creating a SingleStore workspace in the Cloud Portal. To deploy a Self-Managed cluster instead, refer to SingleStore's Deploy documentation. Once the SingleStore workspace/cluster is Active, you'll need the following connection configuration parameters to connect to Fivetran:
  • Host
  • Port
  • Username
  • Password
  • Database
  • Table
• A Fivetran account with a role having the Create Connector permission.

Setup instructions

To authorize Fivetran to connect to your SinlgeStore database, follow these instructions:

Configure SingleStore

1. Configure your firewall and/or other access control systems to allow incoming connections to your SingleStore instance from Fivetran's IPs for your region.

2. Ensure that the SingleStore database user has the SELECT permission.

3. Enable support of OBSERVE queries:

   SET enable_observe_queries=1
   

4. (Optional) Configure snapshots_to_keep and snapshot_trigger_size engine variables. At some point, offsets will be considered stale, meaning the connector will no longer be able to retrieve data associated with that logical point in the WAL (Write-Ahead Log). In practical terms, offsets become stale once they are older than the oldest snapshot in the system. The snapshots_to_keep and snapshot_trigger_size variables control the number and size of snapshots providing you with some control over the data retention window. If you increase snapshots_to_keep or snapshot_trigger_size, offsets will become stale later. If offsets become stale, the connector will be unable to continue streaming change events, and the only way to resolve this is to re-sync all data.

   SET GLOBAL snapshot_trigger_size=10737418240
   SET GLOBAL snapshots_to_keep=4
   

Finish Fivetran configuration

1. Log in to your Fivetran account.
2. Go to the Connectors page, and then click + Add connector.
3. Select SingleStore as the connector type.
4. In the connector setup form, enter the following connection configuration parameters for you SingleStore workspace/cluster:
   • Host
   • Port
   • Database
   • Table
   • Username
   • Password
5. (Optional) Enable SSL and specify related configuration parameters.
6. (Optional) Specify additional Driver Parameters. Refer to The SingleStore JDBC Driver documentation for a list of supported parameters.
7. Click Save & Test.

Setup tests

Fivetran performs the following SingleStore connection tests:

• The Connection test checks if Fivetran can connect to your SingleStore cluster using credentials provided in the setup form
• The Table test checks if specified table exists

(Optional) Post-setup changes

We recommend to adjust the sync frequency in Fivetran so that syncs run more often. This will reduce the likelihood of offsets becoming stale and help avoid the need for a full re-sync.

1. Go to the Setup tab of the connector details page.
2. Change Sync frequency to 15 minutes.

Related articles

description Connector Overview

account_tree Schema Information

settings API Connector Configuration

home Documentation Home