March 2021

Ad Reporting Data Model

Fivetran has created a dbt Core*-compatible ad reporting model that aggregates data from our most popular advertising connectors into a unified table. This pre-built model allows you to compare how your ad spend converts into clicks and impressions on each advertising platform.

The ad reporting model supports the following connectors:

• Facebook Ads
• Google Ads
• LinkedIn Ad Analytics
• Microsoft Ads
• Pinterest Ads
• Twitter Ads

You do not need to have all six advertising connectors to use this model, though you should have at least two.

Learn more in our ad reporting model documentation.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

Broken Connector Sync Schedule

We now schedule the future sync attempts of a connector that is broken for more than 14 days to run every 24 hours. We have added a new broken connector warning to your connector dashboard. Once you resolve the task, the connector will run according to your configured sync schedule.

Dashboard Improvements

Authentication

For improved security, we have changed the browser session expiration policy for the Fivetran dashboard. You will be automatically logged out after one day of inactivity.

Connector and Destination Group IDs

We now display Fivetran Connector ID in the Connection Details section on the Setup tab of the connector dashboard.

We now display Destination Group ID in the Connection Details section of the destination dashboard.

Connector List

The counts in each of the connector list filters are now calculated in accordance with the overall filter settings.

Data Volume

In your connector dashboard's sync bar, the Data Volume field now displays the total size of the uncompressed data that Fivetran processes during every sync. Previously, this field displayed the compressed data size.

The Data Volume value in your dashboard may not match the source data’s actual size. Fivetran adds metadata columns and may use a different data type to store the data in the destination.

If you sync large amounts of data or if you use our File connectors, you will observe large differences in the data volume because the data size of compressed and uncompressed data differs significantly. For example, if you are syncing 204 MB data, the volume of the compressed data will be 19 MB, whereas the volume of the uncompressed data will be 306 MB.

Compressed Data Volume:

Uncompressed Data Volume:

Schema Page

We redesigned the Schema page to make it easier for you to manage schemas, tables, and columns. We enabled table grouping, improved searching, and simplified working with the schema. We also implemented some significant performance improvements, so the table list will render very quickly, even if the number of tables is large.

IMPORTANT: We are gradually rolling out these dashboard improvements for our connectors.

We updated the Schema page design for our Magic Folder connectors. We now display a sync status icon for every file to indicate its sync status: success, warning, or error.

Diagnostic data access

We now ask you to grant Fivetran access to your data for a 21-day period when we need to troubleshoot or fix your broken connector or destination. You can allow or deny the diagnostic data access as well as revoke it at any time before the access period has expired.

Support Portal Update

We've updated our Support Portal to make it easier to find articles that are relevant to you. We've redesigned our landing pages and created more granular sections to help you quickly find answers to your questions.

We've also updated the layout and design to make it easier to navigate our support articles, feature requests, and support tickets:

Transformations for dbt Core

Customizable target name

You can now customize the name of the dbt target that your dbt jobs use. Add the dbt target in the optional targetName field in the deployment.yml file. Learn more about the deployment.yml file in our dbt setup guide.

dbt 0.17.2 support ended

We no longer support dbt 0.17.2.

dbt 0.19.0 support

We now support dbt 0.19.0. You can use this version to run your projects. You can upgrade your existing projects by following dbt's migration guide.

Increased thread limit

We've increased the maximum number of threads that can be used by dbt jobs from 4 to 8.

New connectors

Google Drive

Google Drive is a cloud-based file storage and synchronization service.

Read our Google Drive connector documentation.

Schema changes

Asana

We have added a new field, liked, to replace the hearted field in the TASK table. Asana has deprecated the hearted field. We will deprecate the hearted field from the TASK table after May 31, 2021. Be sure to update your queries.

We will deprecate the assignee_status field to support Asana's breaking changes after May 31, 2021. Be sure to update your queries.

Coupa

We have fixed a bug where the attributes of the EXPENSE_MILEAGE_VEHICLE_TYPE table were added to the separate EXPENSE_TRIP_MILEAGE table. We no longer add the attributes of the EXPENSE_MILEAGE_VEHICLE_TYPE table to the EXPENSE_TRIP_MILEAGE table.

We have renamed the following tables in the Schema tab of your connector details page to align them with the source table names:

• CHARGES to CHARGE
• PROJECT_BUSINESS_GROUP_RELATION to PROJECT_BUSINESS_GROUP_MAPPING
• PROJECT_USER_RELATION to PROJECT_USER_MAPPING
• SHIPPING_TERM_BUSINESS_GROUP_RELATION to SHIPPING_TERM_BUSINESS_GROUP_MAPPING
• UOM to UNIT_OF_MEASURE

These name changes will not impact your destination tables.

Eloqua

We have added two new tables, CONTACT_LIST and CONTACT_LIST_CONTACT, to support Contact list endpoints.

We have added the following new tables to support Contact segments endpoints:

• SEGMENT
• CONTACT_FILTER_SEGMENT_ELEMENT
• CONTACT_FILTER
• CONTACT_LIST_SEGMENT_ELEMENT
• CONTACT_SEGMENT

Fivetran Platform Connector

We have added a new table, CONNECTOR_TYPE, to capture a connector's release phase status.

We have added a new column, _fivetran_deleted, to the USER table to track when a user is deleted.

We have added a new column, _fivetran_deleted, to the CONNECTOR table to track when a connector is deleted.

Freshdesk

We have added the following new lookup tables to map the custom priority, source, and status values in the TICKET table:

• PRIORITY_LOOKUP
• SOURCE_LOOKUP
• STATUS_LOOKUP

HubSpot

We have added two new tables, TICKET_PIPELINE and TICKET_PIPELINE_STAGE to support HubSpot's pipelines object.

Jira

We now sync entries related to Jira permissions. We are gradually rolling out this feature to all existing connectors. Once this feature is available for your connector, you will see the following items in your destination:

• The PERMISSION, PERMISSION_SCHEME, PERMISSION_HOLDER, PROJECT_ROLE tables
• The permission_scheme_id column in the PROJECT table See Jira Schema Information section for details.

We have added a new column, _fivetran_deleted, to the FIELD table. Now we capture custom fields deleted from source.

We now sync the Security Level field.

We now sync Issue Security Schemes. We are gradually rolling out this feature to all existing connectors. Once this feature is available for your connector, you will see the SECURITY_SCHEME, SECURITY_LEVEL, and SECURITY_SCHEME_LEVEL tables in your destination.

Lever

We now sync the CONTACT table.

Pendo

We have added a new custom field, property_*, to the FEATURE_EVENT table.

We have added a new field, is_core_event, to the FEATURE_HISTORY table.

Pipedrive

We have added two new tables, PERSON_PHONE_MAPPING and PERSON_EMAIL_MAPPING, to map the phone and email columns in the PERSON table.

We will deprecate the email and phone columns from the PERSON table after June 30, 2021. Be sure to update your queries.

Recharge

We have added the following new tables:

• CHECKOUT
• CHECKOUT_LINE_ITEM
• METAFIELD
• PAYMENT_SOURCE
• PRODUCT

NOTE: We use webhooks to retrieve data from the Checkouts endpoint. See Recharge's API Documentation for more information.

Recurly

We have added two new columns, started_with_gift and converted_at, to the SUBSCRIPTION table. See Recurly's lookup subscription documentation for more information.

Sage Intacct

We have added a new table, SODOCUMENTENTRY.

Shopify

We have added the _fivetran_deleted column to the USER table.

We have added a new column, is_account_owner, to replace the account_owner column in the USER table. We will deprecate the account_owner column from the USER table after June 30, 2021. Be sure to update your queries.

We have added a new column, client_details_user_agent, to the ORDER table. See Shopify's Order documentation for more information.

Square

We have added a new table, SHIFTS. See Square's documentation on search shifts for more information.

We have added two new tables, CASH_DRAWER and CASH_DRAWER_EVENT. See Square's documentation on cash drawers for more information.

We have added two new tables, DISPUTES and DISPUTES_EVIDENCE. See Square's documentation on Disputes for more information.

We have added six new tables:

• LOYALTY_PROGRAM
• LOYALTY_PROGRAM_ACCRUAL_RULE
• LOYALTY_PROGRAM_REWARD_TIER
• LOYALTY_ACCOUNT
• LOYALTY_ACCOUNT_MAPPING
• LOYALTY_REWARD

See Square's documentation on Loyalty for more information.

Stripe

We have added the following new columns to the ACCOUNT table:

• capabilities_afterpay_clearpay_payments
• capabilities_au_becs_debit_payments
• capabilities_bacs_debit_payments
• capabilities_bancontact_payments
• capabilities_cartes_bancaires_payments
• capabilities_eps_payments
• capabilities_fpx_payments
• capabilities_giropay_payments
• capabilities_grabpay_payments
• capabilities_ideal_payments
• capabilities_jcb_payments
• capabilities_oxxo_payments
• capabilities_p24_payments
• capabilities_sepa_debit_payments
• capabilities_sofort_payments
• capabilities_tax_reporting_us_1099_k
• capabilities_tax_reporting_us_1099_misc

We have added the following new columns to the tables below:

• payment_method_id to the CHARGE table

• discount_id to the ORDER_ITEM and ORDER_REFUND_ITEM tables

• payment_intent_id to the REFUND table

We have added two new columns, destination_payment_id and source_transaction_id, to the TRANSFER table.

We will deprecate the destination_payment and source_transaction columns of the TRANSFER table after June 20, 2021. Be sure to update your queries.

We have added the following new columns to the CARDHOLDER table:

• spending_controls_allowed_categories
• spending_controls_blocked_categories
• spending_controls_spending_limits
• spending_controls_spending_limits_currency

We have added a new column, amount_details_atm_fee, to the AUTHORIZATION and TRANSACTION tables.

We have added a new column, wallet_type, to the CARD table.

Improvements

Adobe Analytics

The Adobe Analytics connector now fetches data using multiple breakdowns without any limitations. Previously, we used a top limit value for dimensions, and the dimension selection order played a crucial role in determining how we sync your data. Note that your syncs now may take longer.

BigQuery

We no longer enforce the column names to be in lower case. We now ignore the case of the column names in your destination tables. Your queries and scripts will continue to execute because BigQuery is case-insensitive.

NOTE: If you have previously synced a column name with upper case letters to your destination table and if a column data type changes in the table, we will rename all the columns in the table to lower case.

We now support JSON flattening. This feature is not enabled by default because we don't support automatic data type promotion from STRING to JSON. See our documentation for more information. If you'd like to enable the feature on your destination, contact our support team. After we enable the feature, you must use your connector's dashboard to drop and re-sync the tables. JSON columns consume more computation resources, and you may observe longer sync durations for the tables.

We now support tables that are partitioned based on a DATETIME column.

NOTE: We only support time partitioning based on daily (DAY) granularity.

Databricks

Our Databricks destination is now generally available.

Read our Databricks destination documentation.

Facebook Ads

We have released pre-built, dbt Core-compatible data models for Facebook Ads. Find the models in Fivetran's dbt hub or data models documentation. Learn more about our dbt Core integration in our Transformations for dbt Core documentation*.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

We have upgraded our Facebook Ads connector from version 7.0 to 9.0 of the Facebook Marketing API and handled deprecations and updates. Learn more about the deprecations and changes in Facebook Marketing API v8.0 changelog and Facebook Marketing API v9.0 changelog.

Freshdesk

We now automatically sync the CONVERSATION table if you include the parent TICKET table in the sync. You can no longer select or deselect the CONVERSATION table in the Schema tab of your connector details page.

Front

We have added a new field, author_id, to the MESSAGE table.

Greenhouse

We have released pre-built, dbt Core-compatible data models for Greenhouse. Find the models in Fivetran's dbt hub or data models documentation. Learn more about our dbt Core integration in our Transformations for dbt Core documentation*.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

HubSpot

We now re-sync the TICKET table if we find that the table hasn't been synced in the last 24 hours because of the Tickets API limitations. We re-sync the table to ensure data integrity.

Intercom

For all new Intercom connectors created after March 1, we will capture webhook events through the endpoint registered in our OAuth app. This change makes it simpler and faster to set up an Intercom connector. For existing Intercom connectors, we will continue to capture webhook events through the Fivetan endpoint you registered in Intercom during setup.

Mailchimp

The Mailchimp connector now supports table blocking for all Mailchimp tables. You can now choose which tables to sync and which to block from syncing to your destination. On your Connector Details page, go to the Schema tab and select or deselect the tables.

If you want to sync a child table, you must include the parent table in the sync.

MySQL

Data syncs no longer fail without explanation when we encounter prohibited DATETIME values (for example, 29 Feb 2021). How we respond to prohibited values varies by sync type:

• Initial syncs: The sync fails when it encounters a prohibited value. We create a Task in your Fivetran dashboard with instructions on how to resolve the issue.

• Incremental updates: The sync does not fail when it encounters a prohibited value. We process the prohibited value and create a Warning in your Fivetran dashboard with instructions on how to resolve the issue. The way we process the prohibited value depends on your sync mode:

  • Soft delete mode: We ignore the prohibited value.
  • History mode: If the prohibited value is a primary key, we ignore it. If the prohibited value is not a primary key, we change it to NULL.

Learn more about Tasks and Warnings in our Alerts documentation.

Sailthru

The Sailthru connector now supports table blocking for all Sailthru tables. You can now choose which tables to sync and which to block from syncing to your destination. On your Connector Details page, go to the Schema tab and select or deselect the tables.

If you want to sync a child table, you must include the parent table in the sync.

Salesforce

We have fixed a bug where deleted records were missing from the OpportunitySplit object and were not being synced from Salesforce's Get Deleted endpoint. We will re-import the OpportunitySplit object based on our re-import logic. We are gradually rolling out this improvement to all existing connectors. The re-imported OpportunitySplit object will count towards your monthly active rows. If you observe that deleted records are not marked as deleted in the OpportunitySplit object, contact our support team.

Shopify

The Shopify connector now supports priority-first sync. Priority-first syncs fetch your most recent data first so that it's quickly ready for you to use. Shopify's priority-first syncs fetch the past 30 days' worth of data from the following tables:

• ABANDONED_CHECKOUT
• COLLECTION
• CUSTOMER
• DRAFT_ORDER
• ORDER
• PRICE_RULE
• PRODUCT
• TENDER_TRANSACTION

To learn more, see our priority-first sync documentation.

Space delimeter support

We have added support for space (\s) in the Delimiter field of your connector setup form. You can now use a single space as a delimiter.

Square

We have updated our OAuth scopes to support the Cash Drawer, Loyalty, and Labor-Shift endpoints of the Square API. You must re-authorize the connector. On your Connector Details page, go to the Setup tab, click Edit connection details, and then click Re-authorize connection. See Square's documentation for more information.

REST API Improvements

Custom Daily Sync Times in API

Now, you can set the time your daily syncs start using the daily_sync_time request parameter for the Modify a Connector endpoint.

Facebook Ads

We now support the timeframe_months field for Facebook Ads connectors created using the Create a Connector endpoint. Learn more about Facebook Ads connector management in our REST API documentation.

New Field Added in API Response

We now display the name_in_destination field for each schema, table, and column in the response for the following endpoints:

• Retrieve a connector schema config
• Retrieve source table columns config
• Reload a connector schema config
• Modify a connector schema config
• Modify a connector database schema config
• Modify a connector table config
• Modify a connector column config

New REST API Endpoints

We have implemented new API endpoints for our REST API:

Destination Management

• Run destination setup tests

Certificate Management

• Approve a destination certificate
• Approve a destination fingerprint

You can also approve certificates and fingerprints automatically by specifying true as a value of the new trust_certificates and trust_fingerprints parameters in the existing Destination Management API endpoints:

• Create a destination
• Modify a destination