March 2021

Ad Reporting Data Model

Fivetran has created a dbt Core*-compatible ad reporting model that aggregates data from our most popular advertising connectors into a unified table. This pre-built model allows you to compare how your ad spend converts into clicks and impressions on each advertising platform.

The ad reporting model supports the following connectors:

Facebook Ads
Google Ads
LinkedIn Ad Analytics
Microsoft Ads
Pinterest Ads
Twitter Ads

You do not need to have all six advertising connectors to use this model, though you should have at least two.

Learn more in our ad reporting model documentation.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

Broken Connector Sync Schedule

We now schedule the future sync attempts of a connector that is broken for more than 14 days to run every 24 hours. We have added a new broken connector warning to your connector dashboard. Once you resolve the task, the connector will run according to your configured sync schedule.

Dashboard Improvements

Authentication

For improved security, we have changed the browser session expiration policy for the Fivetran dashboard. You will be automatically logged out after one day of inactivity.

Connector and Destination Group IDs

We now display Fivetran Connector ID in the Connection Details section on the Setup tab of the connector dashboard.

connector

We now display Destination Group ID in the Connection Details section of the destination dashboard.

warehouse

Connector List

The counts in each of the connector list filters are now calculated in accordance with the overall filter settings.

Data Volume

In your connector dashboard's sync bar, the Data Volume field now displays the total size of the uncompressed data that Fivetran processes during every sync. Previously, this field displayed the compressed data size.

The Data Volume value in your dashboard may not match the source data’s actual size. Fivetran adds metadata columns and may use a different data type to store the data in the destination.

If you sync large amounts of data or if you use our File connectors, you will observe large differences in the data volume because the data size of compressed and uncompressed data differs significantly. For example, if you are syncing 204 MB data, the volume of the compressed data will be 19 MB, whereas the volume of the uncompressed data will be 306 MB.

Compressed Data Volume:

compressed-data

Uncompressed Data Volume:

uncompressed-data

Schema Page

We redesigned the Schema page to make it easier for you to manage schemas, tables, and columns. We enabled table grouping, improved searching, and simplified working with the schema. We also implemented some significant performance improvements, so the table list will render very quickly, even if the number of tables is large.

schema-grouping

schema-tables

We are gradually rolling out these dashboard improvements for our connectors.

We updated the Schema page design for our Magic Folder connectors. We now display a sync status icon for every file to indicate its sync status: success, warning, or error.

magic-folder-schema

Diagnostic data access

We now ask you to grant Fivetran access to your data for a 21-day period when we need to troubleshoot or fix your broken connector or destination. You can allow or deny the diagnostic data access as well as revoke it at any time before the access period has expired.

Support Portal Update

We've updated our Support Portal to make it easier to find articles that are relevant to you. We've redesigned our landing pages and created more granular sections to help you quickly find answers to your questions.

We've also updated the layout and design to make it easier to navigate our support articles, feature requests, and support tickets:

updated-portal

Transformations for dbt Core

Customizable target name

You can now customize the name of the dbt target that your dbt jobs use. Add the dbt target in the optional targetName field in the deployment.yml file. Learn more our deployment.yml guide.

dbt 0.17.2 support ended

We no longer support dbt 0.17.2.

dbt 0.19.0 support

We now support dbt 0.19.0. You can use this version to run your projects. You can upgrade your existing projects by following dbt's migration guide.

Increased thread limit

We've increased the maximum number of threads that can be used by dbt jobs from 4 to 8.

New connectors

Google Drive

Google Drive is a cloud-based file storage and synchronization service.

Read our Google Drive connector documentation.

Schema changes

Asana

We have added a new field, liked, to replace the hearted field in the TASK table. Asana has deprecated the hearted field. We will discontinue the hearted field from the TASK table after May 31, 2021. Be sure to update your queries.

We will discontinue the assignee_status field to support Asana's breaking changes after May 31, 2021. Be sure to update your queries.

Coupa

We have fixed a bug where the attributes of the EXPENSE_MILEAGE_VEHICLE_TYPE table were added to the separate EXPENSE_TRIP_MILEAGE table. We no longer add the attributes of the EXPENSE_MILEAGE_VEHICLE_TYPE table to the EXPENSE_TRIP_MILEAGE table.

We have renamed the following tables in the Schema tab of your connector details page to align them with the source table names:

CHARGES to CHARGE
PROJECT_BUSINESS_GROUP_RELATION to PROJECT_BUSINESS_GROUP_MAPPING
PROJECT_USER_RELATION to PROJECT_USER_MAPPING
SHIPPING_TERM_BUSINESS_GROUP_RELATION to SHIPPING_TERM_BUSINESS_GROUP_MAPPING
UOM to UNIT_OF_MEASURE

These name changes will not impact your destination tables.

Eloqua

We have added two new tables, CONTACT_LIST and CONTACT_LIST_CONTACT, to support Contact list endpoints.

We have added the following new tables to support Contact segments endpoints:

SEGMENT
CONTACT_FILTER_SEGMENT_ELEMENT
CONTACT_FILTER
CONTACT_LIST_SEGMENT_ELEMENT
CONTACT_SEGMENT

Fivetran Platform Connector

We have added a new table, CONNECTOR_TYPE, to capture a connector's release phase status.

We have added a new column, _fivetran_deleted, to the USER table to track when a user is deleted.

We have added a new column, _fivetran_deleted, to the CONNECTOR table to track when a connector is deleted.

Freshdesk

We have added the following new lookup tables to map the custom priority, source, and status values in the TICKET table:

PRIORITY_LOOKUP
SOURCE_LOOKUP
STATUS_LOOKUP

HubSpot

We have added two new tables, TICKET_PIPELINE and TICKET_PIPELINE_STAGE to support HubSpot's pipelines object.

Jira

We now sync entries related to Jira permissions. We are gradually rolling out this feature to all existing connections. Once this feature is available for your connector, you will see the following items in your destination:

The PERMISSION, PERMISSION_SCHEME, PERMISSION_HOLDER, PROJECT_ROLE tables
The permission_scheme_id column in the PROJECT table See Jira Schema Information section for details.

We have added a new column, _fivetran_deleted, to the FIELD table. Now we capture custom fields deleted from source.

We now sync the Security Level field.

We now sync Issue Security Schemes. We are gradually rolling out this feature to all existing connections. Once this feature is available for your connector, you will see the SECURITY_SCHEME, SECURITY_LEVEL, and SECURITY_SCHEME_LEVEL tables in your destination.

Lever

We now sync the CONTACT table.

Pendo

We have added a new custom field, property_*, to the FEATURE_EVENT table.

We have added a new field, is_core_event, to the FEATURE_HISTORY table.

Pipedrive

We have added two new tables, PERSON_PHONE_MAPPING and PERSON_EMAIL_MAPPING, to map the phone and email columns in the PERSON table.

We will discontinue the email and phone columns from the PERSON table after June 30, 2021. Be sure to update your queries.

Recharge

We have added the following new tables:

CHECKOUT
CHECKOUT_LINE_ITEM
METAFIELD
PAYMENT_SOURCE
PRODUCT

We use webhooks to retrieve data from the Checkouts endpoint. See Recharge's API Documentation for more information.

Recurly

We have added two new columns, started_with_gift and converted_at, to the SUBSCRIPTION table. See Recurly's lookup subscription documentation for more information.

Sage Intacct

We have added a new table, SODOCUMENTENTRY.

Shopify

We have added the _fivetran_deleted column to the USER table.

We have added a new column, is_account_owner, to replace the account_owner column in the USER table. We will discontinue the account_owner column from the USER table after June 30, 2021. Be sure to update your queries.

We have added a new column, client_details_user_agent, to the ORDER table. See Shopify's Order documentation for more information.

Square

We have added a new table, SHIFTS. See Square's documentation on search shifts for more information.

We have added two new tables, CASH_DRAWER and CASH_DRAWER_EVENT. See Square's documentation on cash drawers for more information.

We have added two new tables, DISPUTES and DISPUTES_EVIDENCE. See Square's documentation on Disputes for more information.

We have added six new tables:

LOYALTY_PROGRAM
LOYALTY_PROGRAM_ACCRUAL_RULE
LOYALTY_PROGRAM_REWARD_TIER
LOYALTY_ACCOUNT
LOYALTY_ACCOUNT_MAPPING
LOYALTY_REWARD

See Square's documentation on Loyalty for more information.

Stripe

We have added the following new columns to the ACCOUNT table:

capabilities_afterpay_clearpay_payments
capabilities_au_becs_debit_payments
capabilities_bacs_debit_payments
capabilities_bancontact_payments
capabilities_cartes_bancaires_payments
capabilities_eps_payments
capabilities_fpx_payments
capabilities_giropay_payments
capabilities_grabpay_payments
capabilities_ideal_payments
capabilities_jcb_payments
capabilities_oxxo_payments
capabilities_p24_payments
capabilities_sepa_debit_payments
capabilities_sofort_payments
capabilities_tax_reporting_us_1099_k
capabilities_tax_reporting_us_1099_misc

We have added the following new columns to the tables below:

payment_method_id to the CHARGE table
discount_id to the ORDER_ITEM and ORDER_REFUND_ITEM tables
payment_intent_id to the REFUND table

We have added two new columns, destination_payment_id and source_transaction_id, to the TRANSFER table.

We will discontinue the destination_payment and source_transaction columns of the TRANSFER table after June 20, 2021. Be sure to update your queries.

We have added the following new columns to the CARDHOLDER table:

spending_controls_allowed_categories
spending_controls_blocked_categories
spending_controls_spending_limits
spending_controls_spending_limits_currency

We have added a new column, amount_details_atm_fee, to the AUTHORIZATION and TRANSACTION tables.

We have added a new column, wallet_type, to the CARD table.

Improvements

Adobe Analytics

The Adobe Analytics connector now fetches data using multiple breakdowns without any limitations. Previously, we used a top limit value for dimensions, and the dimension selection order played a crucial role in determining how we sync your data. Note that your syncs now may take longer.

BigQuery

We no longer enforce the column names to be in lower case. We now ignore the case of the column names in your destination tables. Your queries and scripts will continue to execute because BigQuery is case-insensitive.

If you have previously synced a column name with upper case letters to your destination table and if a column data type changes in the table, we will rename all the columns in the table to lower case.

We now support JSON flattening. This feature is not enabled by default because we don't support automatic data type promotion from STRING to JSON. See our documentation for more information. If you'd like to enable the feature on your destination, contact our support team. After we enable the feature, you must use your connector's dashboard to drop and re-sync the tables. JSON columns consume more computation resources, and you may observe longer sync durations for the tables.

We now support tables that are partitioned based on a DATETIME column.

We only support time partitioning based on daily (DAY) granularity.

Databricks

Our Databricks destination is now generally available.

Read our Databricks destination documentation.

Facebook Ads

We have released pre-built, dbt Core-compatible data models for Facebook Ads. Find the models in Fivetran's dbt hub or data models documentation. Learn more about our dbt Core integration in our Transformations for dbt Core documentation*.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

We have upgraded our Facebook Ads connector from version 7.0 to 9.0 of the Facebook Marketing API and handled deprecations and updates. Learn more about the deprecations and changes in Facebook Marketing API v8.0 changelog and Facebook Marketing API v9.0 changelog.

Freshdesk

We now automatically sync the CONVERSATION table if you include the parent TICKET table in the sync. You can no longer select or deselect the CONVERSATION table in the Schema tab of your connector details page.

Front

We have added a new field, author_id, to the MESSAGE table.

Greenhouse

We have released pre-built, dbt Core-compatible data models for Greenhouse. Find the models in Fivetran's dbt hub or data models documentation. Learn more about our dbt Core integration in our Transformations for dbt Core documentation*.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

HubSpot

We now re-sync the TICKET table if we find that the table hasn't been synced in the last 24 hours because of the Tickets API limitations. We re-sync the table to ensure data integrity.

Intercom

For all new Intercom connections created after March 1, we will capture webhook events through the endpoint registered in our OAuth app. This change makes it simpler and faster to set up an Intercom connection. For existing Intercom connections, we will continue to capture webhook events through the Fivetan endpoint you registered in Intercom during setup.

Mailchimp

The Mailchimp connector now supports table blocking for all Mailchimp tables. You can now choose which tables to sync and which to block from syncing to your destination. On your Connector Details page, go to the Schema tab and select or deselect the tables.

If you want to sync a child table, you must include the parent table in the sync.

MySQL

Data syncs no longer fail without explanation when we encounter prohibited DATETIME values (for example, 29 Feb 2021). How we respond to prohibited values varies by sync type:

Initial syncs: The sync fails when it encounters a prohibited value. We create a Task in your Fivetran dashboard with instructions on how to resolve the issue.
Incremental updates: The sync does not fail when it encounters a prohibited value. We process the prohibited value and create a Warning in your Fivetran dashboard with instructions on how to resolve the issue. The way we process the prohibited value depends on your sync mode:
- Soft delete mode: We ignore the prohibited value.
- History mode: If the prohibited value is a primary key, we ignore it. If the prohibited value is not a primary key, we change it to NULL.

Learn more about Tasks and Warnings in our Alerts documentation.

Sailthru

The Sailthru connector now supports table blocking for all Sailthru tables. You can now choose which tables to sync and which to block from syncing to your destination. On your Connector Details page, go to the Schema tab and select or deselect the tables.

If you want to sync a child table, you must include the parent table in the sync.

Salesforce

We have fixed a bug where deleted records were missing from the OpportunitySplit object and were not being synced from Salesforce's Get Deleted endpoint. We will re-import the OpportunitySplit object based on our re-import logic. The re-imported OpportunitySplit object will count towards your MAR. If you observe that deleted records are not marked as deleted in the OpportunitySplit object, contact our support team.

Shopify

The Shopify connector now supports priority-first sync. Priority-first syncs fetch your most recent data first so that it's quickly ready for you to use. Shopify's priority-first syncs fetch the past 30 days' worth of data from the following tables:

ABANDONED_CHECKOUT
COLLECTION
CUSTOMER
DRAFT_ORDER
ORDER
PRICE_RULE
PRODUCT
TENDER_TRANSACTION

To learn more, see our priority-first sync documentation.

Space delimeter support

We have added support for space (\s) in the Delimiter field of your connector setup form. You can now use a single space as a delimiter.

Square

We have updated our OAuth scopes to support the Cash Drawer, Loyalty, and Labor-Shift endpoints of the Square API. You must re-authorize the connector. On your Connector Details page, go to the Setup tab, click Edit connection details, and then click Re-authorize connection. See Square's documentation for more information.

REST API Improvements

Custom Daily Sync Times in API

Now, you can set the time your daily syncs start using the daily_sync_time request parameter for the Update a connection endpoint.

Facebook Ads

We now support the timeframe_months field for Facebook Ads connections created using the Create a connection endpoint. Learn more about Facebook Ads connector management in our REST API documentation.

New Field Added in API Response

We now display the name_in_destination field for each schema, table, and column in the response for the following endpoints:

New REST API Endpoints

We have implemented new API endpoints for our REST API:

Destination Management

Run destination setup tests

Certificate Management

You can also approve certificates and fingerprints automatically by specifying true as a value of the new trust_certificates and trust_fingerprints parameters in the existing Destination Management API endpoints: