August 2022

Dashboard Improvements

SAML - Log in directly from login page

SAML users can now log in to their Fivetran accounts directly from the login page. If SSO is enabled for your account, you’ll enter your email on the login page, then continue logging in with SSO. If you want to log in with a password instead, you can switch to the email/password login option.

History mode in beta

Fivetran history mode is now in beta for the following connectors:

Learn more in our history mode documentation.

Transformations for dbt Core

Custom Scheduling

We have released custom scheduling for Transformations for dbt Core*. Transformations with a custom schedule run according to the schedule you choose, except when it overlaps with the upstream connector's schedule. In that case, we wait until the connector finishes syncing to run the transformation. With custom scheduling, you can:

Save on warehouse costs by running your transformation less frequently than your connector.
Ensure that all upstream connectors have synced when you run your transformation by matching its schedule to your slowest upstream connector.

Learn more in our custom scheduling documentation.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

Data lineage graphs with self-service onboarding

We have merged the integrated scheduling and data lineage graph experiences in Transformations for dbt Core, as well as added self-service onboarding. You can now immediately view your unified pipeline and access your integrated schedules. Previously, you had to request that these features be added to your account.

How we implement this change varies based on whether you use Transformations for dbt Core today:

If you're a current Transformations for dbt Core user, we provide an optional upgrade path that automatically creates new integrated transformations from your connected dbt project.
If you're not a current Transformations for dbt Core user, once you connect your dbt project to Fivetran, we enable integrated scheduling and data lineage graphs by default.

Learn more in our integrated scheduling and data lineage graph documentation.

dbt Core versions 1.1.2 and 1.2.0 support

We now support dbt Core 1.1.2 and 1.2.0. You can use these versions to run your projects.

New connectors

Azure Service Bus

Azure Service Bus is a cloud messaging service.

Read our Azure Service Bus connector documentation.

High-Volume Agent Oracle

Oracle is an object-relational database management system created by the Oracle Corporation. Fivetran uses a high-volume agent to replicate data from your Oracle source database and loads it into your destination.

Read our High-Volume Agent Oracle documentation.

SAP Business ByDesign

SAP Business ByDesign is a cloud enterprise resource planning software for small and medium-sized organizations.

Read our SAP Business ByDesign connector documentation.

Schema changes

Adjust

We have added the following new tables:

SK_EVENT
SK_INSTALL
SK_QUALIFIER

We only sync the records whose sk timestamp is on or after June 28, 2022, to these tables.

We have added a new column, creative_name, to the following tables:

ATT_STATUS_UPDATE_IOS
AD_REVENUE
AD_SPEND
CLICK
ERASED_USER_GDPR
IMPRESSION
IN_APP_EVENT
INSTALL
REATTRIBUTION
REATTRIBUTION_REINSTALL
REINSTALL
SAN_ClICK
SAN_IMPRESSION
SESSION
SUBSCRIPTION
UNINSTALL
UPDATE

Amplitude

We have changed the composite primary key of the EVENT table. The new composite key is a combination of the server_upload_time, event_id, and device_id parameters.

Criteo

We have added a new report, PLACEMENT_REPORT.

Facebook Ads

We have added two new breakdowns, app_id and skan_conversion_id.

We have added two new tables, AD_IMAGE_HISTORY and AD_VIDEO_HISTORY. For more details, see Facebook's /adimages and /advideos endpoints documentation.

GitHub

We have added a new table, BRANCH_COMMIT_RELATION, that stores information about branch and commit mapping.

We are gradually rolling out this feature to all existing connectors. Once this feature is available, you will see the BRANCH_COMMIT_RELATION table on the Schema tab. To fetch data for all existing branches and commits, re-sync your connector.

We have added a new table, REPOSITORY_TOPIC, that stores information about repository topics.

We are gradually rolling out this feature to all existing connectors. Once this feature is available, you will see the REPOSITORY_TOPIC table on the Schema tab.

Google Search Console

We now sync the query column instead of the keyword column in the following reports:

KEYWORD_SITE_REPORT_BY_PAGE
KEYWORD_SITE_REPORT_BY_SITE
KEYWORD_PAGE_REPORT

This change only applies to connectors created after August 11, 2022.

Jira

We have excluded the SPRINT table from the BOARD group of tables in the Schema tab of your connector. We sync all sprints entirely when you select the BOARD group of tables in the Schema tab.

LinkedIn Ad Analytics

We now sync the following demographic reports:

AD_ANALYTICS_BY_MEMBER_COMPANY
AD_ANALYTICS_BY_MEMBER_COMPANY_SIZE
AD_ANALYTICS_BY_MEMBER_COUNTRY_V2
AD_ANALYTICS_BY_MEMBER_INDUSTRY
AD_ANALYTICS_BY_MEMBER_JOB_FUNCTION
AD_ANALYTICS_BY_MEMBER_JOB_TITLE
AD_ANALYTICS_BY_MEMBER_REGION_V2
AD_ANALYTICS_BY_MEMBER_SENIORITY

We have added the following tables for the demographic report data:

INDUSTRY
FUNCTION
GEO
ORGANIZATION
TITLE
SENIORITY

Optimizely

We no longer sync client activation events (event_type = client_activation).

Snapchat Ads

We have added a new column, video_views_15s, to the campaign reporting tables, ad reporting tables, and ad squad reporting tables. Learn more in our Snapchat Ads Report Tables documentation.

Stripe

We have added a new column, invoice_item_id, to the INVOICE_LINE_ITEM table.

Taboola

We have added the following new columns to the CONTENT_PROVIDER_PLATFORM_REPORT, CONTENT_PROVIDER_COUNTRY_REPORT, CAMPAIGN_SITE_DAY_REPORT, COUNTRY_REPORT, and PLATFORM_REPORT tables:

cpa_actions_num_from_clicks
cpa_actions_num_from_views
visible_impressions
conversions_value
cpa_clicks
cpa_views
cpa_conversion_rate_clicks
cpa_conversion_rate_views
vctr
vcpm

We have added the following new columns to the TOP_CAMPAIGN_CONTENT_REPORT table:

visible_impressions
actions_num_from_clicks
actions_num_from_views
conversions_value
cpa_clicks
cpa_views
vctr
vcpm
cvr_clicks
cvr_views

Twitter Ads

We have added a new column, budget_optimization, to the CAMPAIGN_HISTORY table to support Twitter API version 11.

Zendesk Support

We have added a new column, html_body, to the TICKET_COMMENT table.

Improvements

Amazon Ads

Our Amazon Ads connector is now generally available. Read our Amazon Ads connector documentation.

App reporting model

Fivetran has created a dbt Core*-compatible app reporting model that aggregates data from our most popular app platform connectors into a unified table. This pre-built model allows you to compare your app activity by device, country, OS version, traffic source, and subscription levels.

The app reporting model supports the Apple App Store and Google Play connectors.

Learn more in our app reporting model documentation.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

Apple App Store

We have released pre-built, dbt Core*-compatible data models for Apple App Store. Find the models in Fivetran's dbt hub. Learn more about our dbt Core integration in our Transformations for dbt Core documentation.

*dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

Box

We have added the Merge Mode functionality to our Box connector. For connectors created after August 25, 2022, you can now opt to sync multiple files into a single table in your destination. In the connector setup form, select your sync strategy as Merge Mode or Magic Folder. For more information, see our setup instructions.

Coupa

We now support the OAuth 2.0 authentication. If you are using the API Key authentication, you need to migrate to OAuth 2.0 before December 1, 2022. You can migrate to OAuth 2.0 by editing the connection parameters and providing the Client ID and Client Secret instead of the API Key. See our Coupa Setup Guide for details about how to obtain them.

Databricks

Our Databricks destination now supports the Unity Catalog feature. For more information, see our setup instructions.

To upgrade your existing tables and schemas to Unity Catalog, see the following:

Google Drive

We have added the Merge Mode functionality to our Google Drive connector. For connectors created after August 24, 2022, you can now opt to sync multiple files into a single table in your destination. In the connector setup form, select your sync strategy as Merge Mode or Magic Folder. For more information, see our setup instructions.

Google Play

We have released pre-built, dbt Core*-compatible data models for Google Play. Find the models in Fivetran's dbt hub. Learn more about our dbt Core integration in our Transformations for dbt Core documentation.

*dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

Kustomer

The following tables now support history mode for connectors created on or after August 01, 2022:

CONVERSATION_HISTORY
CONVERSATION_CHANNEL_HISTORY
CONVERSATION_TAG_HISTORY
CONVERSATION_SUGGESTED_TAG_HISTORY
CONVERSATION_ASSIGNED_TEAM_HISTORY
CONVERSATION_ASSIGNED_USER_HISTORY

MongoDB

Our MongoDB Replica Set connector now supports the following:

Connection tags with SRV hosts. For example, mongodb+srv://server.example.com/?readPreference=secondary&readPreferenceTags=nodeType:ANALYTICS.
Connection strings. For example, mongodb://mongodb0.example.com:27017,mongodb1.example.com:27017,mongodb2.example.com:27017/?replicaSet=myRepl&readPreference=secondary.

For more information, see our setup instructions. We are gradually rolling out this feature to all existing connectors. If you'd like to enable this feature on your connector, contact our support team.

We have added a new database setup test to our MongoDB Replica Set connector. Now, we verify if we can access your MongoDB change streams or oplog.

NetSuite SuiteAnalytics

We have added support for CLOB data type. We now convert the data to STRING data type in your destination tables. We are gradually rolling out this feature to all existing connectors.

Our dbt-Core* compatible data models for NetSuite SuiteAnalytics now support the NetSuite2.com data source. Find the models in Fivetran's dbt hub or data models documentation. Learn more about our dbt Core integration in our Transformations for dbt Core documentation*.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

Salesforce

We have upgraded our Salesforce connector from version 54.0 to 55.0 of the Salesforce REST API. We are gradually rolling out this new feature to all existing connectors.

We have added a new system table, FIVETRAN_FORMULA_MODEL. We now sync Salesforce's formula field translations to the FIVETRAN_FORMULA_MODEL table. The new table doesn't impact the functionality of the FIVETRAN_FORMULA and FIVETRAN_FORMULA_HISTORY system tables.

Salesforce Marketing Cloud

We have decreased the priority fetch period from 14 to 5 days for all the priority-first sync tables to improve sync speeds and reduce the duration of the first forward sync. For more information, see our priority-first sync documentation.

We have decreased the priority fetch period from 5 days to 1 day for all the priority-first sync tables to improve sync speeds and reduce the duration of the first forward sync. For more information, see our priority-first sync documentation.

Snowflake

We have added support for the XML data type. We convert XML data type to VARIANT data type in your Snowflake destination. You can now use XML functions on the VARIANT data type columns to extract and process XML data.

We consider XML as the subtype of JSON. If we receive a JSON value in an XML column from the source, we write the data as VARIANT in your Snowflake destination. We then convert all the existing XML values to JSON strings. To extract the values, you must use the PARSE_XML function on the XML data.

For example, if we receive the following XML value from the source, <root></root>, we process and store the data in Snowflake as <root></root>. When we convert the value to JSON, the value looks like XML in string format, "\n<root></root>".

However, if we receive any other data type value for the same column from the source, we store the data as TEXT in your Snowflake destination.

Twitter Ads

We have upgraded our Twitter Ads connector from version 10 to version 11 of the Twitter API.

Twitter Organic

We have upgraded our Twitter Organic connector from version 10 to version 11 of the Twitter API.

Zoho CRM

We now capture deletes for the following child tables:

PURCHASE_ORDER_PRODUCT
INVOICE_PRODUCT
EVENT_PARTICIPANT
SALES_ORDER_PRODUCT
QUOTE_PRODUCT

We are gradually rolling out this feature to all existing connectors. If you'd like to enable this feature on your connector, contact our support team.

We now support Subforms. We re-import the subform tables. We are gradually rolling out this feature to all existing connectors. If you'd like to enable this feature on your connector, contact our support team.

REST API Improvements

Additional icon URLs

We have added a new icons field to the Retrieve source metadata and Retrieve connector configuration metadata endpoint responses. This field contains a list of additional icon URLs in different formats (for example, .svg or .png) for the connector.

Webhook endpoint check

We have added a new check that we perform during the webhook creation or activation. The endpoint specified in the payload should exist and return a successful HTTP code. Otherwise, we will return an HTTP 400 Bad Request error.

Webhooks retry algorithm

We have changed the retry algorithm for sending webhooks. We now use an exponential backoff strategy - we wait 6 minutes before the first retry, 21 minutes before the second retry, and so on. Learn more in our Retries documentation.