November 2022

Dashboard Improvements

Improved navigation now in beta

We have redesigned the Fivetran dashboard to make it easier to manage your connectors, transformations, and account. With the new dashboard layout, you can do the following:

• Manage your entire Fivetran account from one place
• See the status of all your connectors across multiple destinations
• Navigate your account more easily

We are rolling out this beta feature to most accounts. Use the toggle in the left menu of your Fivetran account to opt in to the improved dashboard. You can switch it back at any time during the beta period.

Timezone support for US daylight saving time

You can now choose to automatically update your time zone to match daylight saving time in the US (from the second Sunday in March to the first Sunday in November). This automatic update ensures that Fivetran begins your sync schedule at the same time we did before the time change.

Learn more in our Destinations documentation.

Lite Connectors

Lite connectors are now available for the following sources:

• Freightview
• Lattice
• Slack

Transformations for dbt Core

Environment variables support

We now support environment variables for Transformations for dbt Core users. You can use environment variables to access private packages, as shown in dbt's Git Token Method documentation.

Learn how to add environment variables in our Scheduled in Fivetran setup guide or Scheduled in Code setup guide.

Manually run and cancel dbt models

You can now manually run, pause, resume, and cancel your dbt models and tests.

New connectors

High-Volume Agent Db2 for i

Db2 for i is a Relational Database Management System integrated with IBM i.

Read our High-Volume Agent Db2 for i connector documentation.

High-Volume Agent SQL Server

SQL Server is a relational database management system created by the Microsoft Corporation. Fivetran uses a high-volume agent to replicate data from your SQL Server source database and loads it into your destination. Read our High-Volume Agent SQL Server connector documentation.

Schema changes

Amazon Ads

We have migrated from version 2 to version 3 of the Bid Recommendations endpoint of the Amazon Ads Sponsored Products API.

We have added the following tables:

• AD_GROUP_THEME_BASED_BID_RECOMMENDATION
• AD_GROUP_BID_RECOMMENDATION_CLICK
• AD_GROUP_BID_RECOMMENDATION_ORDER
• AD_GROUP_BID_RECOMMENDATION_VALUE
• ASIN_THEME_BASED_BID_RECOMMENDATION
• ASIN_BID_RECOMMENDATION_CLICK
• ASIN_BID_RECOMMENDATION_ORDER
• ASIN_BID_RECOMMENDATION_VALUE

We have deprecated the AD_GROUP_BID_RECOMMENDATION and KEYWORD_BID_RECOMMENDATION tables.

BigCommerce

We have added two new tables, PRODUCT_OPTION_CONFIG and PRODUCT_VARIANT_OPTION_CONFIG.

Braze

We have added a new table, CANVAS_EXPERIMENT_STEP_ENTRY_EVENT. If you want to sync data for this table, contact our support team.

We have added the following new columns to the EMAIL_EVENT table:

• bounce_reason
• esp
• from_domain
• ip_pool
• is_amp
• is_drop
• link_alias
• link_id
• machine_open

Eloqua

We have added two new columns, fax and mobile_phone, to the CONTACT table. To sync your historical data from the newly added columns, contact our support team.

HubSpot

For HubSpot connectors created on or after November 11, 2022, we have made the following schema changes:

• We no longer support the following tables:

  • CONTACT_IDENTITY_PROFILE
  • CONTACT_MERGE_AUDIT
  • IDENTITY_PROFILE_IDENTITY

  NOTE: The property_hs_calculated_merged_vids column of the CONTACT table stores the contact merge data.

• In the CONTACT_FORM_SUBMISSION table, we:

  • no longer support the conversion_id, page_url, and title columns.
  • have added a new column, page_id.
  • have changed the primary key. The new composite primary key is a combination of the contact_id, timestamp and form_id columns. Previously, the primary key was a combination of the contact_id and conversion_id columns.

• In the CONTACT table, we no longer support the merged_vids and canonical_vid columns.

• In the PROPERTY_OPTION table, we no longer support double_data and read_only columns.

• In the FORM table, we've added a new column form_type.

Iterable

We have added a new column, additional_properties, to the USER_DEVICE_HISTORY table. If you want to sync the historical data for this column, contact our support team.

Jira

We have added a new column, state, to the SPRINT table.

Kafka

We have added new columns to support serialized keys. Learn more in our Schema information documentation.

Shopify

We have added a new column, checkout_id, to the ORDERS table.

TikTok Ads

We have added the following new tables:

• AD_REPORT_LIFETIME
• ADGROUP_REPORT_LIFETIME
• CAMPAIGN_REPORT_LIFETIME

Workday HCM

We have added a new table, PERSONAL_INFORMATION_ETHNICITY.

We have added a new column, employee_type, to the POSITION table. Trigger a re-sync of this table to backfill the data for this column.

We have added a new column, usage_type, to the COMMUNICATION_USAGE_DATA table. Trigger a re-sync of this table to backfill the data for this column.

Improvements

ActiveCampaign

You can now configure your ActiveCampaign connector using the Fivetran REST API. This feature is available only for Free, Standard, Enterprise, and Business Critical accounts.

Azure Service Bus

You can now configure your Azure Service Bus connector using the Fivetran REST API. The feature is available only for Free, Standard, Enterprise, and Business Critical accounts.

BigQuery destination

BigQuery restricts us from dropping multiple columns of a table. We have implemented a temporary workaround and can now drop multiple columns from a BigQuery table.

We recreate a table if we face a reserved column name failure during a DROP COLUMN operation. We drop a table column using the following steps:

1. Create a copy of the original table.
2. Rename the copy table to the original table.
3. Drop the original table.
4. Drop the column from the copy table.

Coupa

We have fixed a bug in our Coupa connectors that use OAuth2.0 authentication, where we didn't sync the INVENTORY_TRANSACTION table. To sync the INVENTORY_TRANSACTION table, select the following read scopes for the Client ID when configuring the connector:

• core.inventory.transfer.read
• core.inventory.consumption.read
• core.inventory.adjustment.read
• core.inventory.return_to_supplier.read

For existing connectors using OAuth2.0, go to your Coupa portal and add the above scopes to your Client ID.

Databricks

We have added automatic deployment cloud detection to the Databricks setup form. We now use your Host details to detect your Databricks deployment cloud automatically. For example, during the destination setup, if you enter the Server Hostname as xxx.cloud.databricks.com, we detect that the Databricks deployment cloud is AWS. If we can auto-detect the cloud provider successfully, the Databricks Deployment Cloud field will not be visible in the setup form.

Files

We have added support for the deprecated Parquet INT96 data type to support legacy parquet files that use INT96 data type for timestamps.

Fivetran Platform Connector

We now sync dbt transformations' user-facing logs into the LOG table.

Gainsight Customer Success

We now re-import all tables without timestamp fields. Previously, we re-imported specific tables. Learn more in our re-import documentation.

Google Ad Manager

We have added a new field, Ad unit view, to the connector setup form. Now, you can choose Top Level, Flat, or Hierarchical ad unit view.

Google Ads

We have changed the way we sync geo targets. Previously, we only synced geo targets that were bound to campaign criteria. Now, we sync all geo targets twice per month on the 1st and 15th days of the month.

Google Analytics 4

Our Google Analytics 4 connector is now generally available.

Read our Google Analytics 4 connector documentation.

HubSpot

We now name custom object association tables using the format, [FROM_SCHEMA_NAME]_TO_[TO_SCHEMA_NAME].

The composite primary keys for the custom object association tables are the from_id, to_id, and type fields.

We will re-sync all the existing custom object association tables to reflect the changes.

We now use version HubSpot API v3 to sync the following parent tables and their associated child tables:

• DEAL
  • DEAL_STAGE
  • DEAL_CONTACT
  • DEAL_COMPANY
  • DEAL_PROPERTY_HISTORY
  • MERGED_DEAL
• CONTACT
  • CONTACT_PROPERTY_HISTORY
  • CONTACT_FORM_SUBMISSION
• COMPANY
  • COMPANY_PROPERTY_HISTORY

New HubSpot connectors created on or after November 11, 2022, will support this feature by default. We are gradually rolling out this improvement to all existing connectors.

Klaviyo

Now, if we persistently receive the 500 Internal Server Error for the Events endpoint from Klaviyo, we only skip the relevant period that returns this error and continue syncing data for the EVENT table. Previously, the entire sync failed.

MongoDB

We now support the configuration of the read preference during the connector setup using connection tags in the connection string. For more information, see either the MongoDB sharded cluster or the MongoDB replica set setup guide.

PostgreSQL

We now sync incremental data from partitioned tables into the parent table. This change applies to new connectors created on or after November 8, 2022. For connectors created before November 8, 2022, we sync incremental data from partitioned tables into partitions. If you want to sync incremental data into the parent table, contact Fivetran Support.

SAP Business By Design

Our SAP Business By Design connector is now generally available.

Read our SAP Business By Design documentation.

SFTP

You can now connect your SFTP Server to Fivetran using an SSH tunnel. For more information, see our SFTP setup guide.

Webhooks

We now support writing the collected webhook events to your Azure Blob Storage and Google Cloud Storage containers in JSON format as a permanent store. For more information, see our Webhooks Setup Guide.

REST API Improvements

AWS Lambda

You can now specify secrets as key-pair values when you configure the Lambda connector using our REST API.

LinkedIn Ad Analytics

In the LinkedIn Ad Analytics REST API configuration, we have removed the 90-day option from the postClickAttributionWindowSize and viewThroughAttributionWindowSize fields because data older than 30 days cannot be updated.

Zendesk Support

Now, you can configure the API usage limit of the Zendesk Support connector using our REST API. Previously, the default API usage limit was 90%. For more information, see our Zendesk Support API Configuration documentation.