October 2018

Destination primary key changes

For the following destinations, Fivetran now drops the primary_key field from existing tables in your destination if there is a primary_key coming in from your data source:

• MySQL
• PostgreSQL
• SQL Server
• Redshift

Faster BigQuery loading

We’ve made two major updates to the way we load data into BigQuery:

• We are now using the interactive priority method for our queries instead of the batch priority method. Despite being designed for Fivetran’s batch update use-case, the BigQuery team has informed us that the batch priority method simply queued up our jobs without any actual improvement in the write performance.

• We have also re-enabled BigQuery's SQL tool to use the ‘MERGE’ function. This reduces the data manipulation language (DML) statements per load from two to one.

As a result of these updates, the data-writing process is more than five times faster than it was previously.

Logs

Bug fix

We have fixed a bug that caused incomplete failure records in our user-facing logs. Previously, when connection tests failed because of a failed sync, our logs only showed the connection test failure. Now both failures are accurately recorded.

Format changes

Our user-facing logs now include a UUID-format incident identifier, exception_id, that gives our support and developer teams access to more detailed information about connector issues.

Log events

Fivetran now supports flushing dashboard activity event data to supported log services.

Each log service has a unique path for this data:

New connectors

Adobe Analytics

Adobe Analytics lets you analyze data about customer engagement with your website or application.

Read our Adobe Analytics connector documentation.

Iterable

Iterable is a growth marketing and customer engagement platform. It enables brands to create, execute and optimize campaigns across email, push, SMS, in-app and more with unparalleled data flexibility.

Read our Iterable connector documentation.

Schema changes

Greenhouse

The timestamp column now supports two new datetime formats:

• UTC (For example, '2018-06-27 23:51:40 UTC')
• Java TimeZone (For example, '2018-07-03T22:49:22.689Z')

We have renamed the id column to tag_id in the CANDIDATE_TAG table.

We have changed the primary key columns for the following tables:

• candidate_id and index are the new primary keys for the following tables:

  • ADDRESS
  • ATTACHMENT
  • EMAIL_ADDRESS
  • PHONE_NUMBER
  • SOCIAL_MEDIA_ADDRESS
  • WEBSITE_ADDRESS

• scorecard_id and index are the new primary keys for the following tables:

  • SCORE_CARD_ATTRIBUTE
  • SCORE_CARD_QNA

• application_id and index are the new primary keys for the APPLICATION_QNA table.

• job_post_id and index are the new primary keys for the JOB_POST_QUESTION table.

Snapchat Ads

We have added a new column, operation, to the AD_SQUAD_SEGMENT_HISTORY table.

Xero

We have added two new columns, edition and class, to the ORGANIZATION table.

Improvements

LinkedIn Ad Analytics

You can now sync all campaigns for a specific ad account without having to select them manually in our setup form.

MongoDB

We have optimized the way we do incremental updates from the oplog of nested updates. Previously, when you updated deeply nested fields, we had to fetch the entire document using collection.find({_id:?}). This process can be extremely slow and delay syncs for hours, especially when deeply nested fields are frequently updated. We can now fetch these documents in batches, making our MongoDB syncs ten times faster.

NetSuite SuiteAnalytics

We have released service version 3 of the NetSuite SuiteAnalytics connector, which includes the following new features:

• Tables that can't be incrementally updated can now be re-imported. The frequency with which we update these re-import tables is based on how fast we can ingest them from NetSuite. For more information, see our NetSuite SuiteAnalytics guide.
• The ITEM_LOCATION_MAP re-import table will update every sync. To see the duration of each table import, you can refer to the table warning field on the Schema tab of your Fivetran dashboard's Connector page.
• We can now upload records for tables where the primary key column contains null values. For more information, see our NetSuite SuiteAnalytics guide.

Pinterest Ads

We no longer process ITEM reports for advertisers who are not running shopping campaigns, because Pinterest only allows advertisers who are running shopping campaigns to access these reports.

PostgreSQL

We have clarified the wording of the rescheduled sync notification message in your Sync History.

Segment

The Segment connector now handles delete event types. We create a new table, DELETE, in your destination to track these events.