March 2023

Dashboard Improvements

Select or deselect all connector schemas

You can now select or deselect all schemas at the same time on your connector's Schema tab.

Fivetran AWS Tokyo Servers

We have added new servers in the AWS Asia Pacific (Tokyo) region.

If you're on a Business Critical plan, you can now select this region in your destination setup form. Connectors configured in a destination run in their designated cloud and region, meaning that your data doesn't leave our region-specific servers during processing. For example, if you configure your connectors to use our servers in Japan, your data does not leave Japan during processing.

If you're on an Enterprise or Business Critical plan, this is the default region if you choose Japan as your data processing location and AWS as your cloud provider.

We now support AWS Private Link connections to sources and destinations in the ap-northeast-1 region.

For more information, see our data residency documentation.

Fivetran AWS US GovCloud Servers

We have added new servers in the AWS GovCloud US West region.

If you're on a Business Critical plan, you can now select this region in your destination setup form. Connectors configured in a destination run in their designated cloud and region, meaning that your data doesn't leave our region-specific servers during processing. For example, if you configure your connectors to use our servers in GovCloud (US West), your data does not leave GovCloud (US West) during processing.

We now support AWS Private Link connections to sources and destinations in the AWS GovCloud (US West) region.

For more information, see our data residency documentation.

Lite Connectors

Lite connectors are now available for the following sources:

• Close
• Lessonly
• Lob
• Monday.com
• Open Exchange Rates
• Wrike

S3 Data Lake

We now support data lakes built on Amazon S3 as destinations. We use AWS Glue as the data catalog for your destination tables.

Read our S3 Data Lake documentation.

Transformations for dbt Core

dbt Core version 1.4.1 support has been deprecated.

We have deprecated support for dbt Core* 1.4.1 due to a known bug. You can still run existing projects using dbt Core 1.4.1 until we implement the support for newer versions.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

dbt Core version 1.4.5 support

We now support version 1.4.5 of dbt Core. You can use this version to run your projects.

NOTE: For this version of dbt Core, we don't support SQL Server, Azure Synapse, and MySQL. These destinations aren't currently supported by custom dbt Core adapters.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

New connectors

The Trade Desk

The Trade Desk is an advertising service that provides users with a multifaceted approach to advertising automation.

Read our The Trade Desk connector documentation.

Schema changes

ADP Workforce Now

We no longer sync the PERSON and WORK_ASSIGNMENT tables.

Amplitude

We now insert null values for the following columns to support their deprecation:

• amplitude_event_type
• device_brand
• device_manufacturer
• device_model
• is_attribution_event
• user_creation_time

However, these columns continue to exist in the destination schema. For more information on the changes, see Amplitude’s community forum.

BambooHR

We have fixed the issue with the duplicated columns in the EMPLOYEE table displayed on the Schema tab. One duplicate type had the name in proper case (for example, 'Address Line'), the other one had the name in snake case (for example, 'address_line'). If you only enabled columns with the name in proper case, and disabled columns with the name in snake case, that may have resulted in data not synced to your destination. In that case, we recommend that you enable the relevant tables with the name in snake case, and re-sync your connector. The columns with the names in proper case are now unavailable for selection and eventually will be not displayed on the Schema tab.

Coupa

We have added a new table, SUPPLIER_ITEM. We are gradually rolling out this change to all existing connectors.

Facebook Pages

We have added a new metric, post_video_views_60s_excludes_shorter, to the LIFETIME_POST_METRICS_TOTAL table. For more information, see Facebook's API documentation.

GitHub

We have added two new columns, forks_count and watchers_count, to the REPOSITORY table.

Google Ads

We have made the following schema changes to support Google Ads API v13:

• We have added two new reports, TRAVEL_ACTIVITY_GROUP_VIEW and TRAVEL_ACTIVITY_PERFORMANCE_VIEW.

• We have added the following fields to the AD_GROUP_CRITERION resource:

  • ad_group_criterion.listing_group.case_value.activity_country.value
  • ad_group_criterion.listing_group.case_value.activity_id.value
  • ad_group_criterion.listing_group.case_value.activity_rating.value

• We have added the following fields to the ASSET resource:

  • asset.field_type_policy_summaries
  • asset.hotel_property_asset.hotel_address
  • asset.hotel_property_asset.hotel_name

• We have added the asset_set.hotel_property_data.hotel_center_id and asset_set.hotel_property_data.partner_name fields to the ASSET_SET resource.

• We have added the campaign_criterion.local_service_id.service_id field to the CAMPAIGN_CRITERION resource.

• We have added the following fields to the AD_GROUP_AD report:

  • ad_group_ad.ad.travel_ad
  • ad_group_ad.ad.video_ad.bumper.action_button_label
  • ad_group_ad.ad.video_ad.bumper.action_headline

• We have added the following fields to the AD_GROUP_ASSET report:

  • ad_group_asset.primary_status
  • ad_group_asset.primary_status_details
  • ad_group_asset.primary_status_reasons

• We have added the following fields to the CAMPAIGN report:

  • campaign.hotel_property_asset_set
  • campaign.travel_campaign_settings.travel_account_id
  • campaign.target_cpm.target_frequency_goal.target_count
  • campaign.target_cpm.target_frequency_goal.time_unit

• We have added the following fields to the CAMPAIGN_ASSET report:

  • campaign_asset.primary_status
  • campaign_asset.primary_status_details
  • campaign_asset.primary_status_reasons

• We have added the following fields to the CUSTOMER report:

  • customer.image_asset_auto_migration_done
  • customer.image_asset_auto_migration_done_date_time
  • customer.location_asset_auto_migration_done
  • customer.location_asset_auto_migration_done_date_time

• We have added the following fields to the CUSTOMER_ASSET report:

  • customer_asset.primary_status
  • customer_asset.primary_status_details
  • customer_asset.primary_status_reasons

• We have replaced the ad_group.explorer_auto_optimizer_setting.opt_in field with the ad_group.optimized_targeting_enabled field in AD_GROUP report.

• We have removed the user_list.rule_based_user_list.combined_rule_user_list.* and user_list.rule_based_user_list.expression_rule_user_list.* fields from the USER_LIST resource, since the Google Ads API no longer supports CombinedRuleUserList and ExpressionRuleUserList.

• We have removed the ad_group_criterion.listing_group.case_value.product_bidding_category.country_code field from the AD_GROUP_CRITERION resource and campaign.target_cpm field from CAMPAIGN report, since these fields are no longer supported in the Google Ads API.

Google Analytics 4

We have removed the following metrics from Ecommerce Purchases prebuilt reports, to be consistent with the Google Analytics Data API (GA4) Schema compatibility changes.

• ADD_TO_CARTS
• ECOMMERCE_PURCHASES
• ITEM_VIEWS

Microsoft Lists

We now add a new metadata table, <SITE_NAME>_COLUMNS, to your destination. This metadata table contains the mapping details of the table columns in the destination and the list columns in the Microsoft Lists user interface.

Pendo

We have added a new primary key column, _fivetran_id, to the following tables to avoid data integrity issues:

• GUIDE_EVENT
• POLL_EVENT
• EVENT
• TRACK_TYPE_EVENT

This functionality is enabled by default for new connectors created after March 21, 2023. We are gradually rolling out this change to all the existing connectors.

We now sync the properties field of the PAGE_EVENT and EVENT tables.

Recharge

We have upgraded our Recharge connector from version 2021-01 to 2021-11 of the Recharge API. We have made the following schema changes:

• We have added a new table, CHARGE_TAX_LINE, to replace the tax_lines column of the CHARGE table.
• We have added a new table, PAYMENT_METHODS, to replace the PAYMENT_SOURCE table.
• We have added a new table, PLANS, to replace the PRODUCT table.
• We have added two new columns, purchase_item_id and purchase_item_type, to the ORDER_LINE_ITEM and CHARGE_LINE_ITEM tables.
• We have renamed the following columns of the DISCOUNT table:
  • applies_to_product_type to applies_to_purchase_item_type
  • usage_limit to automatic_redemptions_per_customer
• We have renamed the following columns of the ORDER table:
  • shopify_cart_token to external_cart_token
  • shopify_order_id to external_order_id_ecommerce
  • shopify_order_number to external_order_number_ecommerce
• We have renamed the following columns of the SUBSCRIPTION table:
  • shopify_product_id to external_product_id_ecommerce
  • shopify_variant_id to external_variant_id_ecommerce
• We have renamed the following columns in the CHARGE_LINE_ITEM and ORDER_LINE_ITEM tables:
  • shopify_product_id to external_product_id_ecommerce
  • shopify_variant_id to external_variant_id_ecommerce
• We have renamed the shopify_customer_id column of the CUSTOMER table to external_customer_id_ecommerce.
• We have renamed the shopify_order_id column of the CHARGE table to external_order_id_ecommerce.
• We have renamed the shopify_variant_id column of the ONETIME_PRODUCT table to external_variant_id_ecommerce.
• We have deprecated the status column of the CUSTOMER table.
• We have deprecated the subscription_id column of the ORDER_LINE_ITEM and CHARGE_LINE_ITEM tables.

For more information about the changes, see Recharge API’s changelog.

If you want to sync historical data for these columns, contact our support team.

Salesforce

We have added two new tables, ENTITY_DEFINITION and PUBLISHER.

Twilio

We have added a new table, MESSAGING_SERVICE.

We have added a new column, voice_url, to the INCOMING_PHONE_NUMBER table.

We have added a new column, messaging_service_sid, to the MESSAGE table.

We are gradually rolling out these changes to all existing connectors.

Zendesk Support

We have added two new tables, TICKET_COLLABORATOR and TICKET_FOLLOWER. We are gradually rolling out these changes to all existing connectors.

We now support the TICKET_CUSTOM_STATUS table which captures Custom Ticket Statuses.

Improvements

Amazon S3

We now support syncing PGP (Pretty Good Privacy) encrypted files. We have added a new advanced configuration option, PGP Encryption Options, to the connector setup form. For more information, see our setup instructions.

Amplitude

We can now sync your data from Amplitude sources hosted in the EU region.

AppsFlyer

We can now sync the SKAdNetwork (SKAN) attribution events from Data Locker and Webhooks. We are gradually rolling out this feature to all existing connectors. For more information, see our AppsFlyer documentation.

Azure Blob Storage

We can now sync packed JSON files from Azure Blob Storage. We have added a new drop-down menu, JSON Delivery Mode, to the connector setup form. The drop-down menu provides you the option to choose how Fivetran should handle your JSON data. For more information, see our setup instructions.

Azure Cosmos DB

We now support pack modes so that you can choose the form in which Fiveran delivers your data. Learn more in our pack mode options documentation.

We can now access your data using resource tokens. Resource tokens allow Fivetran to read from specific Azure Cosmos DB resources. Learn more in our resource token documentation.

Azure Monitor

We have renamed our Azure Log Analytics log service to Azure Monitor. This name change applies to the Fivetran dashboard and our documentation.

Read our Azure Monitor documentation.

BambooHR

Our BambooHR connector is now generally available. Read our BambooHR connector documentation.

Braze

You can now opt to create a new Braze connector without configuring a Braze Currents connection. Previously, you could only create a new connector to sync Braze Currents data using a cloud storage service. For more information, see our setup instructions. As a part of this improvement, we will also perform a one-time free re-sync of the following tables:

• CAMPAIGN
• CAMPAIGN_TAG
• CANVAS
• CANVAS_STEP
• CANVAS_STEP_SEQ
• CANVAS_TAG
• CANVAS_VARIATION
• CARD
• CARD_EXTRA
• CARD_TAG
• MESSAGE_VARIATION

Facebook Ads

We can now fetch creative entities with large thumbnail images (up to 600 x 600).

Facebook Pages

We have upgraded our Facebook Pages connector to version 16.0 of the Facebook Graph API. Learn more about the changes in the Facebook Graph API v16.0 changelog.

Freshservice

Our Freshservice connector is now generally available. Read our Freshservice connector documentation.

Google Ads

We have improved how we fetch account labels for the ACCOUNT_LABEL_HISTORY table. Now, we get additional label information using the Customer Label resource. If a customer client doesn't have access to all labels assigned to it, we need to send the request using the manager client ID to increase the privileges. We are gradually rolling out this improvement for all the existing connectors. If you have a lot of accounts with labels, this change may result in a slight increase in MAR.

We now support Google Ads API v13.

Google Play

You can now select and deselect tables to sync from the Schema tab on your connector details page.

Instagram Business

We have upgraded our Instagram Business connector from version 15.0 to 16.0 of the Graph API. Learn more about the changes in the Facebook API v16.0 changelog.

Kantata

We now support Subscribed Events to capture deletes in a more efficient way. For more information, see our Kantata setup guide. We are gradually rolling out this change to all existing connectors.

Lever

We have added the EU Region toggle to the connector setup form. Set the toggle to ON if your application is in the EU region. For more information, see our setup guide.

Mixpanel

We have changed the primary key of the EVENT table to avoid duplicate events from Mixpanel. We have removed the time column and now use only the _fivetran_id column as the primary key. For event records after July 1, 2021, we calculate the _fivetran_id value as the hash of the insert_id, distinct_id, and name columns. New connectors created after March 19, 2023, will support this feature by default. We are gradually rolling out this improvement to all the existing connectors.

NetSuite SuiteAnalytics

The NetSuite2.com data source for our NetSuite SuiteAnalytics connector is now generally available. Read our NetSuite SuiteAnalytics connector documentation.

We now re-import the POSTING_ACCOUNT_ACTIVITY table in every sync.

PostgreSQL

We now support PostgreSQL version 15. Read our PostgreSQL documentation for more information.

We are deprecating the logical replication with the test_decoding plugin update method. We are gradually migrating all users who use this method to logical replication with the pgoutput plugin. If you use the test_decoding method, we generate a warning message in your Fivetran dashboard that contains instructions for migrating to the pgoutput method. Follow the steps in that message to complete your migration. To learn about the differences between the pgoutput method and the test_decoding method, see our logical replication documentation.

We now support the TRUNCATE command for the pgoutput update method. If a table is truncated, the records prior to the TRUNCATE timestamp appear as _fivetran_deleted=true in the destination. If you are using history mode, the truncated table is deleted in your destination, just like a regular delete.

We now sync bounds information for DATERANGE, TSRANGE, and TSTZRANGE data types to the destination, where the values EXCLUSIVE and INCLUSIVE indicate the bounds.

Qualtrics

We have fixed a bug in our Qualtrics connector. Previously, we could not detect some answers for Carry Forward Choices questions and did not sync all the records to the QUESTION_RESPONSE table. To sync all records, we added a new column, question_response.question_option_key, to the QUESTION_RESPONSE table and changed the data type of the question_response.sub_question_key column from INT to STRING.

We are gradually rolling out this improvement to all existing connectors. To rectify previously synced data, trigger a connector re-sync or contact our support team to re-sync the QUESTION_RESPONSE table.

We now automatically capture Carry Forward Choices data in the QUESTION_OPTION and SUB_QUESTION tables.

Recharge

We now capture deletes for the CHARGE and CUSTOMER tables. In order to remove the previously deleted records from these tables, trigger a full historical re-sync for these tables.

Redshift

During UPDATE operations, we now insert null values for the VARBYTE columns that exceed 65535 bytes.

RingCentral

Our RingCentral connector is now generally available. Read our RingCentral connector documentation.

Salesforce Commerce Cloud

You can now configure your Salesforce Commerce Cloud connector using the Fivetran REST API. This feature is available only for Free, Standard, Enterprise, and Business Critical accounts.

Salesloft

Our Salesloft connector is now generally available. Read our Salesloft connector documentation.

Stripe

We have changed our sync strategy to handle invalid or deleted connected accounts. We now display a warning on your connector dashboard when we detect invalid or deleted Stripe connected accounts and skip these accounts from the sync. Previously, we didn't skip these accounts, causing the connector to break.

Twitter Ads

You can now select or deselect the following tables to sync from the Schema tab on your connector details page:

• CAMPAIGN_HISTORY
• CARD_HISTORY
• FUNDING_INSTRUMENT_HISTORY
• LINE_ITEM_HISTORY
• TWEET

Typeform

We have added a new column, choice_text, to the RESPONSE_ANSWER_CHOICE table. We now sync the free-text response of the Other option in multiple choice questions to the choice_text column.

Xero

We have changed the new connector creation process for Xero connectors. We now use a different authentication method for Xero connectors created after March 24, 2023. We use Client ID and Client Secret for authentication. For more information, see our Xero Setup Guide.

We will continue to support the existing Xero connectors. You can also opt to migrate your existing connector to the new authentication method.

Zendesk Support

Our Zendesk Support connector now supports table level re-sync.

REST API Improvements

Change in Databricks Destination Management API

For the service parameter in the API request for our Databricks destination, you now need to specify databricks as a value. The old values databricks_aws and databricks_azure are still valid.

Changed response format in Webhooks resource

We have added the following new fields to the response for our REST API Webhook endpoints:

• code
• message
• data

The data field contains all fields returned in the response in the old format.

An example of the previous response format:

{
  "id": "program_quoth",
  "type": "account",
  "url": "https://your-host.your-domain/webhook",
  "events": [
    "sync_start",
    "sync_end"
  ],
  "active": true,
  "secret": "******",
  "created_at": "2022-04-29T09:41:08.583Z",
  "created_by": "_airworthy"
}

An example of the new response format:

{
  "code": "Success",
  "message": "Account webhook has been created",
  "data": {
    "id": "program_quoth",
    "type": "account",
    "url": "https://your-host.your-domain/webhook",
    "events": [
      "sync_start",
      "sync_end"
    ],
    "active": true,
    "secret": "******",
    "created_at": "2022-04-29T09:41:08.583Z",
    "created_by": "_airworthy"
  }
}

Connector name and specific value returned in response

Fivetran now takes into account all connectors specifics and returns the correct name_in_destination field value for schemas in the response for the following REST API endpoints:

• Retrieve a Connector Schema Config
• Reload a Connector Schema Config
• Modify a Connector Schema Config
• Modify a Connector Database Schema Config
• Modify a Connector Table Config
• Modify a Connector Column Config

Previously, we returned the schema name only with the generic naming rules applied.