September 2023

Apache Kafka destination

You can now use AWS MSK as your destination with Fivetran connectors. For more information, see our setup instructions.

Lite Connectors

Lite connectors are now available for the following sources:

OneLake

We have added OneLake as one of our supported destinations. Read our OneLake documentation.

Organization Management

We have released the beta version of Organization Management. Organization Management provides a central control point for all Fivetran accounts within your organization. For more information, see our Organization Management documentation.

Transformations for dbt Core

dbt Core version 1.6.3 support

We now support dbt Core version 1.6.3. You can use this version to run your projects.

NOTE: For this version of dbt Core, we don't support SQL Server, Azure Synapse, and MySQL destinations. These destinations are not currently supported by custom dbt Core adapters.

We now support MySQL or MariaDB destinations for dbt Core versions 1.1.*.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

dbt logs visual improvements

In order to make dbt run logs easy to read, we have updated the font and added color coding for certain statuses.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

New dbt Quickstart packages

We have added new Quickstart packages for the following connectors:

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

Schema changes

Amazon Selling Partner

We have added two new tables, FINANCIAL_SHIPMENT_EVENT_ITEM and FINANCIAL_SHIPMENT_EVENT_ITEM_PROMOTION.

We have added a new column, event_type, to the FINANCIAL_AFFORDABILITY_EXPENSE_EVENT and FINANCIAL_SHIPMENT_EVENT tables.

We have added a new column, inventory_summary_id, to the FBA_INVENTORY_RESEARCHING_QUANTITY_ENTRY table.

We have added the following new columns to the FINANCIAL_CHARGE_COMPONENT table:

charge_kind
linked_to
linked_to_id

We have added the following new columns to the FINANCIAL_FEE_COMPONENT table:

fee_kind
linked_to
linked_to_id

We have added a new primary key column, _fivetran_id, to the FBA_INVENTORY_SUMMARY and FINANCIAL_SAFET_REIMBURSEMENT_ITEM tables.

We have deprecated the financial_event_id and financial_event_type columns in the FINANCIAL_CHARGE_COMPONENT and FINANCIAL_FEE_COMPONENT tables.

We are gradually rolling out these changes to all existing connectors.

We have deprecated the ORDER_ITEM_APPROVAL_STATUS and ORDER_ITEM_APPROVAL_TYPE tables because the Orders API v0 has deprecated the ItemApprovalTypes and ItemApprovalStatus attributes.

Brevo

We have added two new tables, EMAIL_CAMPAIGN_STATS and EMAIL_STATISTICS, as child tables of the EMAIL_CAMPAIGN table.

Clickup

We have added two new tables, SUB_TASK and SUB_TASK_ASSIGNEE.

Facebook Ads

We have removed the status_processing_progress and status_value fields from the AD_VIDEO_HISTORY table as the structure of the video_status object has changed and the corresponding API document has not been updated. We will add columns to align with the new structure as soon as the Facebook document is updated.

Fivetran Platform Connector

We now populate the created_at column as null in the DESTINATION table if the is_active column is false.

We have added two new columns, is_active and type, to the DESTINATION table.

GitHub

We have added the repository_id column as a primary key in the SECURITY_ALERT table.

We have added a new column, payload, to the DEPLOYMENT table.

HubSpot

We have added the following new tables to support the HubSpot Web Analytics API:

GEOLOCATION_ANALYTICS_DAILY_REPORT
GEOLOCATION_ANALYTICS_MONTHLY_REPORT
GEOLOCATION_ANALYTICS_OVERALL_REPORT
GEOLOCATION_ANALYTICS_WEEKLY_REPORT
SESSIONS_ANALYTICS_DAILY_REPORT
SESSIONS_ANALYTICS_MONTHLY_REPORT
SESSIONS_ANALYTICS_OVERALL_REPORT
SESSIONS_ANALYTICS_WEEKLY_REPORT
SOURCES_ANALYTICS_DAILY_REPORT
SOURCES_ANALYTICS_MONTHLY_REPORT
SOURCES_ANALYTICS_OVERALL_REPORT
SOURCES_ANALYTICS_WEEKLY_REPORT
TOTALS_ANALYTICS_DAILY_REPORT
TOTALS_ANALYTICS_MONTHLY_REPORT
TOTALS_ANALYTICS_WEEKLY_REPORT
UTM_ANALYTICS_DAILY_REPORT
UTM_ANALYTICS_MONTHLY_REPORT
UTM_ANALYTICS_OVERALL_REPORT
UTM_ANALYTICS_WEEKLY_REPORT

We are gradually rolling out these changes to all HubSpot connectors.

Kustomer

We have added a new table, CUSTOM_ATTRIBUTE_METADATA, to store the metadata properties for the custom attributes of a specific resource.

Notion

We have renamed the BLOCK table to PARENT_BLOCK and the DATABASE_OBJECT_OPTION table to DATABASE_OBJECT_PROPERTY.

We no longer sync the PAGE_PROPERTY_TYPE table. We now sync the data as custom fields to the PAGE_PROPERTY table.

We no longer sync the properties_* columns in the DATABASE_OBJECT table. We now sync the data from the properties_* columns to the DATABASE_OBJECT_PROPERTY table.

Be sure to update your queries.

Outreach

We have added two new columns relationship_follow_up_sequence_id and relationship_user_id, to the MAILING table.

Pipedrive

We have added the following new columns to the PRODUCT_DEAL table because Pipedrive has changed its logic of product discounts:

discount
discount_type
tax_method

Recharge

We have added a new column, value_type, to the DISCOUNT table.

Snowplow

We have added the collector_tstamp column as a part of the composite primary key of the EVENT table. The new composite primary key is a combination of the collector_tstamp and event_id columns.

We have added a new column, collector_tstamp, to the UA_PARSER_CONTEXT and [CUSTOM_CONTEXT] tables. We have also added the collector_tstamp column to the composite primary key of the two tables. The new composite primary key is a combination of the collector_tstamp and event_id columns.

We are gradually rolling out these changes to all existing connectors.

Workable

We have added a new primary key column, fivetran_id, to the SUPPORTED_FILE_TYPE table. We no longer use the file_type column as the primary key of the SUPPORTED_FILE_TYPE table. Be sure to update your queries.

Workday HCM

We have made the following schema changes for the Workday HCM connectors created before September 9, 2023:

We no longer sync the following columns of the tables below:

TABLE	COLUMNS
`COMPENSATION_GRADE_PROFILE`	`compensation_grade_profile` `index` `job_profile_id`
`JOB_PROFILE_CERTIFICATION`	`certification_id` `country`
`JOB_PROFILE_COMPETENCY`	`id`
`JOB_PROFILE_EDUCATION`	`degree_id`
`JOB_PROFILE_LANGUAGE`	`id`
`JOB_PROFILE_PAY_RATE`	`country` `pay_rate_id`
`JOB_PROFILE_RESTRICT_TO_COUNTRY`	`country` `id`
`JOB_PROFILE_SKILL`	`job_profile_skill_id`
`JOB_PROFILE_WORK_EXPERIENCE`	`id`

We will stop syncing the following tables after September 15, 2023:

CERTIFICATION_SPECIALTY_ACHIEVEMENT
LANGUAGE_ABILITY_PROFILE
UNION_MEMBERSHIP
WORKER_COMPENSATION
WORK_STUDY

Be sure to update your queries.

We have added the following new tables:

COMPENSATION_GRADE
JOB_PROFILE_CERTIFICATION_SPECIALTY
JOB_PROFILE_COMPENSATION_GRADE_PROFLE
JOB_PROFILE_LANGUAGE_ABILITY
JOB_PROFILE_WORKER_COMPENSATION

We have added the following new columns to the tables below:

TABLE	NEW COLUMNS
`JOB_PROFILE`	`compensation_grade_id` `union_code` `union_membership_requirement` `work_study_award_source_code` `work_study_requirement_option_code`
`JOB_PROFILE_CERTIFICATION`	`country_code`
`JOB_PROFILE_COMPETENCY`	`competency_id`
`JOB_PROFILE_LANGUAGE`	`language_id`
`JOB_PROFILE_PAY_RATE`	`country_code`
`JOB_PROFILE_RESTRICT_TO_COUNTRY`	`country_code` `country_id`
`JOB_PROFILE_SKILL`	`skill_code`
`JOB_PROFILE_WORK_EXPERIENCE`	`work_experience_id`

Improvements

Adjust

The Adjust connector now supports priority-first sync. Priority-first syncs fetch your most recent data first so that it's quickly ready for you to use. Adjust's priority-first syncs fetch the most recent 30 days' worth of data from all tables. To learn more, see our priority-first sync documentation.

Adobe Analytics Data Feed

We now support connecting to your Azure Blob storage container using Azure Private Link. Learn more in our setup guide.

Airtable

We now share Airtable webhooks across different Fivetran Airtable connectors. You can now select the same base across more than two connectors. Previously, Airtable webhook limitations allowed to create a maximum of two OAuth webhooks per Airtable base. Each connector created its own webhook, as a result you could only select the same base in a maximum of two Fivetran connectors. We are gradually rolling out this improvement to all existing connectors.

Amazon DynamoDB

We have upgraded the Amazon DynamoDB connectors from AWS SDK 1.12 to AWS SDK 2.17. We are gradually rolling out this improvement to all existing connectors.

Amazon S3

We now support syncing PGP (Pretty Good Privacy) signed and encrypted files. We have added a new option, Signer's Public Key, to the PGP Encryption Options configuration in the connector setup form. For more information, see our setup instructions.

Amazon Selling Partner

The Amazon Selling Partner connector now supports priority-first sync for the FINANCE module. Priority-first syncs fetch your most recent data first so that it's quickly ready for you to use. Amazon Selling Partner's priority-first syncs fetch the most recent 7 days' worth of data from the FINANCE module. To learn more, see our priority-first sync documentation.

Apache Kafka

We now name the Kafka topics and schemas in the following formats:

Topic name format: <schema_name>-<table_name>
Schema name format: <schema_name>.<table_name>

Previously, we used the <schema_name>.<table_name>.<hash> format for both topic and schema names. We made this change to avoid adding a hash and to avoid conflicts, as Kafka does not differentiate between periods (.) and underscores (_) in topic names.

Azure Blob Storage

Azure Data Lake Storage

Our Azure Data Lake Storage (ADLS) destination is now generally available. Read our Azure Data Lake Storage destination documentation.

Azure Monitor

We have updated the pattern for log filters. For more information, see our documentation.

Chargebee Product Catalog 2.0

We no longer capture deletes for the INVOICE and SUBSCRIPTION tables. We no longer update the _fivetran_deleted column in these tables.

Firebase

Firebase connectors created on or after September 29, 2023, support two pack modes, Packed mode and Unpacked mode, at the connector level. Packed mode is the new default option. When you choose a pack mode on the Connector Details page, the selected mode applies to all tables that are enabled for syncs.

We will continue to support both the pack modes at the table level for Firebase connectors created before September 29, 2023. Learn more in our Firebase pack mode documentation.

Fivetran Platform Connector

We now sync HVR 6 usage data to the DESTINATION and USAGE_COST tables.

We now sync all roles to the ROLE and ROLE_PERMISSION tables. Previously, we synced only the custom roles with an associated account.

Gong

We have added a new field, Base URL, to the connector setup form in order to maintain data synchronization after Gong updated the base URL for its API calls. For more information, see our setup instructions.

Google Ads

We have upgraded our Google Ads connector from version 14 to 14.1 of the Google Ads API and added the following metrics and segments to custom reports:

metrics.new_customer_lifetime_value
metrics.all_new_customer_lifetime_value
segments.new_versus_returning_customers

HubSpot

We now perform a weekly table re-sync to capture soft deletes for the CONTACT_LIST_MEMBER table. We are gradually rolling out this feature to all existing connectors.

Klaviyo

The Klaviyo connector now supports priority-first sync. Priority-first syncs fetch your most recent data first so that it's quickly ready for you to use. Klaviyo's priority-first syncs fetch the most recent 7 days' worth of data from the EVENT table. To learn more, see our priority-first sync documentation.

We have upgraded our connector to Klaviyo API v2023-07-15. New Klaviyo connectors created on or after August 22, 2023, will use the new API version. As a part of the upgrade, we have made the following schema changes:

We have added the following new tables:
- CAMPAIGN_MESSAGE
- CAMPAIGN_MESSAGE_SEND_TIME
- CAMPAIGN_TRACKING_UTM_PARAM
- GLOBAL_EXCLUSION
- SEGMENT
- SEGMENT_PERSON
We have added the following new columns to the CAMPAIGN table:
- archived
- scheduled
- send_option_*
- tracking_options_is_*
We have added two new columns, editor_type and text, to the EMAIL_TEMPLATE table.
We have added two new columns, archived and trigger_type, to the FLOW table.
We have added the following new columns to the PERSON table:
- last_event_date
- predictive_analysis_*
- subscriptions_*
We have deprecated the following columns from the tables below:
- is_segmented from the CAMPAIGN table.
- is_writeable from the EMAIL_TEMPLATE table.
- folder_name from the LIST table.
- email from the LIST_EXCLUSION table.
- customer_filter and trigger from the FLOW table.
We have deprecated the INTEGRATION table. We will now sync the data from this table to the METRIC table.
We now sync the message-related columns to the CAMPAIGN_MESSAGE table instead of the CAMPAIGN table to support the Klaviyo API changes.
We have added the list_id column as a primary key for the LIST_EXCLUSION table.
The id column in the LIST_EXCLUSION table now has a foreign key relationship to the PERSON table.

We are gradually migrating all existing connectors to the new API version.

Marketo

We can now provide you with an estimate of your BULK API quota usage by using your Marketo SOAP API credentials. To support this feature, we have made the following changes to the connector setup form:

Added a new toggle, Provide SOAP Credentials for BULK API Quota Estimation. You can opt to set this toggle to ON and provide your SOAP API credentials while configuring the connector.
Made the following fields optional:
- SOAP API User Id
- SOAP API Encryption Key
- SOAP API Endpoint

For more information, see our setup instructions.

NetSuite SuiteAnalytics

We have fixed a bug where we didn't sync some records for the following tables:

ITEM_DEMAND_PLAN_LINES in the NetSuite.com data source when the table contained multiple records with the same primary key value.
revRecScheduleRecurrence in the NetSuite2.com data source when the table didn't have a primary key and contained duplicate records and the same values.

We have added the _fivetran_index column as a primary key column to both the tables. We are gradually rolling out this fix to all existing connectors.

Outreach

We have fixed a bug where we didn't sync some records in the DATA_CONNECTION table. This occurs when records with active data connection have null values in the id column. To fix this issue, we now convert the null values to -1, in the id column.

Pardot

We now capture deletes for the LIST_MEMBERSHIP table. We are gradually rolling out this feature to all existing connectors.

Pipedrive

We now re-import the LEAD_HISTORY table in every sync. Previously, we re-imported this table once a day to capture updates and deletes.

Qualtrics

We have fixed a bug where we didn't accurately capture deletes for the DISTRIBUTION table.

Recharge

The Recharge connector now supports priority-first sync for the following tables:

ADDRESS
ADDRESS_DISCOUNT
ADDRESS_SHIPPING_LINE
CHARGE
CHARGE_DISCOUNT
CHARGE_LINE_ITEM
CHARGE_ORDER_ATTRIBUTE
CHARGE_SHIPPING_LINE
CHARGE_TAX_LINE
CHECKOUT
CHECKOUT_LINE_ITEM
CUSTOMER
METAFIELD
ONE_TIME_PRODUCT
ORDER
ORDER_LINE_ITEM
PAYMENT_METHOD
PLAN
SUBSCRIPTION
SUBSCRIPTION_HISTORY
UTM_TAG
Priority-first syncs fetch your most recent data first so that it's quickly ready for you to use. Recharge's priority-first syncs fetch the most recent 15 days' worth of data from the supported tables. We are gradually rolling out this change to all existing connectors. To learn more, see our priority-first sync documentation.

We now sync the METAFIELD table using the following endpoints:

META_FIELD_CHARGE
META_FIELD_CUSTOMER
META_FIELD_ORDER
META_FIELD_STORE
META_FIELD_SUBSCRIPTION

S3 Data Lake

Our S3 Data Lake destination is now generally available. Read our S3 Data Lake destination documentation.

You can now connect Fivetran to your S3 buckets using an AWS PrivateLink connection. You can opt to use an AWS PrivateLink connection if your S3 bucket and destination are in the same AWS Region. This feature is supported only in Business Critical plans. For more information, see our setup instructions.

Salesforce Commerce Cloud

The Salesforce Commerce Cloud connector now supports syncing custom fields in the following tables:

ADDRESS
CAMPAIGN
CATALOG
CATEGORY
COUPON
COUPON_REDEMPTION
CUSTOMER
GIFT_CERTIFICATE
GIFT_CERTIFICATE_TRANSACTION
ORDER_BILLING_ADDRESS
ORDER_HISTORY
ORDER_PAYMENT_INSTRUMENT
ORDER_PRODUCT_ITEM
ORDER_SHIPMENT
ORDER_SHIPMENT_ADDRESS
ORDER_SHIPPING_ITEM
PAYMENT_INSTRUMENT
PRODUCT
PROMOTION

SAP Concur

Our SAP Concur connector now supports the Company Level authentication mode. For more information, see our setup instructions.

SFTP

Slack

You can now configure your Slack connector using the Fivetran REST API. This feature is available only for Free, Standard, Enterprise, and Business Critical accounts.

Square

We have added the following scopes to our Square connectors:

BANK_ACCOUNTS_READ
DEVICE_CREDENTIAL_MANAGEMENT
GIFTCARDS_READ
INVOICES_READ
MERCHANT_PROFILE_READ
SUBSCRIPTIONS_READ
VENDOR_READ

We require these scopes to access the Subscriptions, Invoices, Gift Cards, Businesses, and Financials endpoints.

Workday HCM

We now support table level re-syncs for the Workday HCM connectors. You can now select or deselect the child tables to sync from the Schema tab on your connector details page. We are gradually rolling out these improvements to all existing connectors. For more information, see our Workday HCM documentation.

Zendesk Support

We now capture deletes for the POST table. To capture deletes, we re-import the table once every 30 days.

REST API Improvements

Azure Data Lake Storage

You can now configure your Azure Data Lake Storage (ADLS) destination using the Fivetran REST API.

Azure Functions

You can now specify secrets as key-pair values when you configure the Azure Functions connector using our REST API.

Google Cloud Functions

You can now specify secrets as key-pair values when you configure the Google Cloud Functions connector using our REST API.

New REST API Endpoints

We have implemented new Group Management API endpoints for our REST API. You can now leverage the following new features: