May 2021

Dashboard Improvements

We've redesigned the add connector screen to make it easier to find the connector you're looking for. We've sorted all connectors by type (databases, marketing analytics, etc.) If you're not sure which category your connector belongs to, you can review the All data sources list or use the search bar at the top of the page.

source-selection

ISO 27001 Certification

Fivetran has received ISO/IEC 27001:2013 certification, recognizing our commitment to the highest level of information security. The certification provides a standardized confirmation for users regarding security best practices and capabilities. For more information, read our blog post.

Security

We will stop support for HTTP and TLS versions 1.1 and 1.0 for our webhooks (webhooks.fivetran.com) and events (events.fivetran.com) endpoints.

Make sure to use only an HTTPS URL with TLS version 1.2 or above when you configure a new connection for the following connectors:

AppsFlyer
Branch
Eloqua
GitHub
Greenhouse
Help Scout
HubSpot
Intercom
Iterable
Jira
Mandrill
Pipedrive
Recharge
Segment
SendGrid
Shopify
Snowplow
SurveyMonkey
Webhooks

We recommend updating your existing connector configurations to use an HTTPS URL with TLS version 1.2 or above for enhanced security.

System Wide Alerts

We have changed the behavior of incident notifications. Now we send notifications about the following general components once System wide alerts is checked on the Notification Settings page:

Dashboard UI
Fivetran API
Replication Servers

system-wide-alerts

We have migrated our existing users over to the new notification settings.

Transformations for dbt Core

dbt 0.18.2 and 0.19.1 support

We now support dbt 0.18.2 and 0.19.1.

Schema changes

Asana

We have added a new field, current_status_color, to the PROJECT table to represent the overall state of the project (green - On Track, yellow - At Risk, red - Off Track, blue - On Hold).

Braintree

We have added a new table, DISPUTE_STATUS_HISTORY, which stores a dispute's status change history. For more information, see Braintree's documentation.

Coupa

We have added a new column, custom_fields, to all the tables except the mapping tables (tables whose names end with the _MAPPING suffix). We will sync the custom fields as JSON objects.

Facebook Pages

We have added three new tables:

DAILY_POST_METRICS_TOTAL
UNIQUE_DAILY_POST_METRICS_TOTAL
DAILY_POST_METRICS_BY_LOCALE

These tables contain daily post video metrics.

Fivetran Platform Connector

We have added a new column, sync_frequency, to the CONNECTOR table.

Jira

We have added a new column, author_id, to the ISSUE_FIELD_HISTORY and ISSUE_MULTISELECT_HISTORY tables.

This column stores the identifiers of the users who made changes to a given Jira issue. This feature applies to all connectors set up after September 10, 2020. Re-sync your existing connector to sync this data for all historical changes.

Lever

We have added a new column, closedAt, to the REQUISITION table.

Shopify

We have added a new table, SHOP, to support Shopify's Shop resource.

We have added a new table, METAFIELD, to support Shopify's Metafield resource. We support the metafields of the following endpoints:

COLLECTION
CUSTOMER
DRAFT_ORDER
ORDER
PRODUCT
PRODUCT_IMAGE
PRODUCT_VARIANT
SHOP

By default, we sync the metafields of the CUSTOMER and SHOP endpoints.

To sync the metafields of other supported endpoints, go to the Schema tab of your connector details page and select the related *_METAFIELD options. For example, to sync the metafields of the ORDER endpoint, select ORDER_METAFIELD.

IMPORTANT: You may observe slow sync speeds if you sync the metafields of the non-default endpoints.

We have added a new table, ORDER_RISK. By default, we exclude the table from your syncs. On your Connector Details page, go to the Schema tab and select the table. For more information, see Shopify's Order Risk documentation.

Stripe

We have renamed the following tables:

TAX_RATE_INVOICE_MAPPING to INVOICE_TAX_RATE
TAX_RATE_INVOICE_ITEM_MAPPING to INVOICE_ITEM_TAX_RATE
TAX_RATE_SUBSCRIPTION_MAPPING to SUBSCRIPTION_TAX_RATE

We will continue to sync the TAX_RATE_*_MAPPING tables for existing connectors.

We have added a new column, price_id, to the INVOICE_LINE_ITEM table.

We have added two new columns, price_id and plan_id, to the INVOICE_ITEM table.

We have added a new column, three_d_secure_usage_supported, to the PAYMENT_METHOD_CARD and CARD tables.

Zendesk Support

We have added a new table, TICKET_LINK.

Improvements

Amplitude

We now obfuscate the apiKey and secretKey values in the connector setup form.

BigQuery

We now support the europe-west1 region as a BigQuery dataset location. In the destination setup form, select EUROPE_WEST_1 as your data location.

Region Name	Region Description
europe-west1	Belgium

We have added support for the following scenarios as part of the JSON flattening feature:

If a JSON column is null, or has an empty array, or is an empty object, we don’t create the column in your destination. For example, if we receive a JSON structure, such as {"json_col":{"num":1, "arr":[]}} or {"json_col":{"num":1, "arr":{}}} or {"json_col":{"num":1, "arr":null}}, then we don't create the json_col.arr column in the BigQuery destination.
BigQuery doesn't support NULL values in an array. If null values are present in an array, we convert the ARRAY type to STRING type. For example, if we receive null values in an array, such as [1, 2, 3, null] or ["name_1", "name_2", "null", null], then we write the data as STRING "[1, 2, 3, null]" or "[\"name_1\", \"name_2\", \"null\", null] to the BigQuery destination.
BigQuery supports arrays, for example, [{"name":"name_1", "nums":[1, 2, 3]}] or [{"name":"name_2", "nums":[11, 22, 33]}], but doesn't support nested arrays (arrays of arrays), such as [[1, 2, 3], [4, 5, 6]]. We convert the nested inner arrays to STRING ["[1, 2, 3]", "[4, 5, 6]"], and then write the data to the BigQuery destination.

Coupa

Our Coupa connector is now generally available.

Read our Coupa connector documentation.

Greenhouse

You can now exclude the EEOC table from your syncs. On your Connector Details page, go to the Schema tab and deselect the table.

HubSpot

We now support incremental updates for the CONTACT_LIST_MEMBER table.

Klaviyo

We have released pre-built, dbt Core-compatible data models for Klaviyo. Find the models in Fivetran's dbt hub or data models documentation. Learn more about our dbt Core integration in our Transformations for dbt Core documentation*.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

Marketo

We now display a broken connector task on your connector dashboard when we detect that your Bulk API daily usage quota is not sufficient for a sync to complete. We keep track of your daily bulk API usage, and if your quota is less than 70% of the estimated API usage required to successfully sync data, we display this error. Increase your Bulk API quota to resolve the task.

We have made the following changes to the Schema tab of your connector details page:

Added the ACTIVITY_TYPE and OPPORTUNITY_MEMBERSHIP tables. You can now select or deselect the tables from your syncs. We include the two tables in the sync if you have enabled the Allow all option in the Schema Change Settings menu.
Fixed the name of the API_USAGE table. Previously, the table was incorrectly named USAGE.
Added the child tables of COMPANY, LEAD, LIST, PROGRAM, and PROGRAM_MEMBERSHIP tables. The child tables are view-only. You must include or exclude the parent tables.

MySQL

To correct data integrity issues, we will now trigger a re-sync for a table when we detect a reduction in the size of one of its SET columns. Before the re-sync begins, you may notice a SET type change detected error in your logs.

When we detect a reduction in the size of a table's ENUM columns, we now create an ENUM type change detected error in your logs before we trigger a re-sync for that table. Previously, we added FIVETRAN_ERROR_ENUM_DELETED values to your ENUM columns before we triggered a re-sync.

Pendo

We have added a new field, Sync Mode, to the connector setup form. You can now select all or specific Pendo App Ids to sync to your destination. See our setup instructions for more details.

Shopify

The Shopify connector now supports table-level re-syncs.

The Shopify connector now supports table blocking for all Shopify tables. You can now choose which tables to sync and which to block from syncing to your destination. On your Connector Details page, go to the Schema tab and select or deselect the tables. If you want to sync a child table, you must include the parent table in the sync.

We now support table grouping in the Schema tab of your connector details page.

Snapchat Ads

We have released pre-built, dbt Core-compatible data models for Snapchat Ads. Find the models in Fivetran's dbt hub or data models documentation. Learn more about our dbt Core integration in our Transformations for dbt Core documentation*.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

We have added Snapchat Ads to our ad reporting dbt model. Learn more in our ad reporting model documentation.

Stripe

We now support the REFUND and DISPUTE tables for Stripe Connect accounts. If you'd like to sync data from Stripe connected accounts, contact our support team.

Webhooks

We have added support for two regional endpoints, European Union (EU) and United States (US). We restrict your data to the selected destination location. We will continue to process the existing connectors' data in the US region. If you want to migrate your data to the EU region, contact our support team.

REST API Improvements

New REST API Endpoints

We have implemented a new Group Management API endpoint for our REST API. You can now leverage the new Delete a group feature.

Support external OAuth credentials to authorize connectors in Public API

We have expanded the public API endpoints Create a Connector and Modify a Connector. Now you can authorize the Asana connector by adding the auth section to payloads.