November 2021

Dashboard Improvements

Connector usage tab

We have added a new Usage tab on the Connector Details page where you can see the connector's MAR consumption. You can filter the MAR consumption data by table. If you have permission to view your account's Usage page, you can also click the Go to account usage link to see the account's total MAR consumption.

Magic Folder Errors and Warnings Filter

We have implemented a new filter option for Magic Folder connectors in the Fivetran dashboard. You can now filter your file list to view only failed files or files with warnings. Select the Only show errors or warnings checkbox to filter your file list.

Unfiltered list:

Filtered list:

Refresh Connector List button

You can now click the Refresh Connector List button on the Connectors page to manually refresh the connector list. We also display the last refreshed time beside the button. Previously, we automatically refreshed the connector list at every change.

New Role-Based Access Control Model

We are gradually migrating to a new role-based access control (RBAC) model. Our new RBAC model provides more granular user role permission management than the legacy RBAC model. A user role now encompasses permissions across all types of Fivetran resources: accounts, destinations, and connectors.

Your user experience may vary depending on your migration status. To find out whether you have been migrated or not, check which of the role sets corresponds to the role set displayed on the Roles tab of the Account Management page in your Fivetran dashboard.

Transformations for dbt Core

dbt 0.21.0 support

We now support dbt 0.21.0. You can use this version to run your projects. You can upgrade your existing projects by following dbt's migration guide.

New connectors

AWS MSK

AWS Managed Streaming for Kafka is a managed distributed streaming platform.

Read our AWS MSK connector documentation.

Azure Event Hubs

Azure Event Hubs is a managed distributed streaming platform.

Read our Event Hubs Now connector documentation.

OneDrive

OneDrive is a cloud-based file storage service.

Read our OneDrive connector documentation.

Schema changes

Asana

We have added a new column, resource_subtype, to the TASK table.

If you want to sync historical data for this column, contact Fivetran Support to re-sync the TASK table.

Eloqua

We have added the following new columns to the CONTACT table:

• address1
• address2
• address3

If you want to sync historical data for these columns, contact Fivetran Support to re-sync the CONTACT table.

Greenhouse

We have added a new column, is_template, to the JOB table.

Outbrain

We have added a new table, PUBLISHER_REPORT, which stores metrics of publishers.

QuickBooks

We have added the following new columns to the INVOICE_LINE table:

• detail_type
• line_num
• id

Salesforce Marketing Cloud

We have added the event_type field as a primary key in the EVENT table. We don't recommend a re-sync because this change resolves a rare discrepancy that may not have affected you and because the EVENT table can take a long time to sync. If you do notice data discrepancies in the EVENT table and must re-sync it, contact Fivetran Support.

Shopify

We have added a new column, body_html, to the PRODUCT and COLLECTION tables.

Snapchat Ads

We now sync the following reporting tables:

• CAMPAIGN_DAILY_REPORT
• CAMPAIGN_UNIQUE_IMPRESSION_DAILY_REPORT
• CAMPAIGN_DEMOGRAPHIC_DAILY_REPORT
• CAMPAIGN_DEVICE_MAKE_DAILY_REPORT
• CAMPAIGN_DEVICE_OS_DAILY_REPORT
• CAMPAIGN_GEO_COUNTRY_DAILY_REPORT
• CAMPAIGN_GEO_DMA_DAILY_REPORT
• CAMPAIGN_GEO_REGION_DAILY_REPORT
• CAMPAIGN_INTEREST_DAILY_REPORT
• AD_DAILY_REPORT
• AD_UNIQUE_IMPRESSION_DAILY_REPORT
• AD_DEMOGRAPHIC_DAILY_REPORT
• AD_DEVICE_MAKE_DAILY_REPORT
• AD_DEVICE_OS_DAILY_REPORT
• AD_GEO_COUNTRY_DAILY_REPORT
• AD_GEO_DMA_DAILY_REPORT
• AD_GEO_REGION_DAILY_REPORT
• AD_INTEREST_DAILY_REPORT
• AD_SQUAD_DAILY_REPORT
• AD_SQUAD_UNIQUE_IMPRESSION_DAILY_REPORT
• AD_SQUAD_DEMOGRAPHIC_DAILY_REPORT
• AD_SQUAD_DEVICE_MAKE_DAILY_REPORT
• AD_SQUAD_DEVICE_OS_DAILY_REPORT
• AD_SQUAD_GEO_COUNTRY_DAILY_REPORT
• AD_SQUAD_GEO_DMA_DAILY_REPORT
• AD_GEO_REGION_DAILY_REPORT
• AD_INTEREST_DAILY_REPORT

Stripe

We have added a new column, invoice_item_id, to the PRICE table.

TikTok Ads

We have added a new column, status, to the CAMPAIGN_HISTORY table.

Twilio

We have added the following new tables:

• SERVICE stores service information
• CONVERSATION stores conversation information
• CONVERSATION_MESSAGE stores conversation message information
• CONVERSATION_USER stores user information
• PARTICIPANT stores participant information
• ROLE stores role information
• ROLE_PERMISSION stores role permission information
• RECEIPT stores delivery receipt information

To sync these tables, select them in the Schema tab on the connector details page in your Fivetran dashboard.

Twitter Organic

We have added a new table, TWITTER_USER_HISTORY, which stores the user data.

Improvements

AppsFlyer

We now use the is_primary_attribution column of the EVENT table for primary key hashing. To generate the hash of the primary key, we use the following logic :

• For an event on or before 2021-11-10T00:00:00Z, we use the existing column list.
• For an event after 2021-11-10T00:00:00Z, we use the existing column list and the is_primary_attribution column.

Databricks

Our Databricks destination now supports clusters with Databricks Runtime versions 9.0 - 10.x.

Google Ad Manager

We have upgraded our Google Ad Manager connector from version 202102 to 202111 of the Ad Manager API and handled deprecations and updates. Learn more about the deprecations and changes in Google Ad Manager's API documentation.

• Removed columns:

  • AD_SERVER_INVENTORY_LEVEL_WITH_CPD_PERCENT_REVENUE
  • TOTAL_INVENTORY_LEVEL_ALL_REVENUE
  • TOTAL_INVENTORY_LEVEL_WITH_CPD_AVERAGE_ECPM

• New dimensions:

  • INVENTORY_SHARE_ASSIGNMENT_ID
  • INVENTORY_SHARE_ASSIGNMENT_NAME
  • INVENTORY_SHARE_OUTCOME

• New columns:

  • TOTAL_VIDEO_MATCHED_OPPORTUNITIES
  • TOTAL_VIDEO_MATCHED_DURATION
  • TOTAL_VIDEO_DURATION
  • TOTAL_VIDEO_BREAK_START
  • TOTAL_VIDEO_BREAK_END

Google Display & Video 360

We now sync dimensions and metrics that were recently introduced in the DoubleClick Bid Manager API.

• New dimensions:

  • Budget Segment Budget
  • Budget Segment End Date
  • Budget Segment Pacing Percentage
  • Budget Segment Start Date
  • Budget Type (Segment)
  • Rejection Reason
  • Insertion Order Goal Type
  • Insertion Order Goal Type
  • Line Item Budget
  • Line Item End Date
  • Line Item Pacing Percentage
  • Line Item Start Date
  • Matched Genre Target
  • OM SDK Capable
  • Vendor Measurement Mode
  • Audibility At Complete
  • Audibility At Start
  • Video Player Size at Completion
  • Video Player Size at First Quartile
  • Video Player Size at Midpoint
  • Video Player Size at Start
  • Video Player Size at Third Quartile
  • Video Resized
  • Video Content Duration
  • Video Content Live Stream

• New metrics:

  • Cookie Consented Floodlight Impressions
  • Cookie Unconsented Floodlight Impressions
  • Duplicate Consented Floodlight Impressions
  • Engagement Rate
  • Targeted Impressions
  • Bid Responses
  • Percentage from Current IO Goal
  • Cookie Unconsented Clicks
  • All YouTube Sequence Impressions
  • Measurable Impressions For Video Player Size
  • Available Requests (Deal)
  • Targeted Impressions (Deal)
  • Available Requests (Line Item)
  • Targeted Impressions (Line Item)
  • Win Rate

See the full list of supported dimensions and metrics in the DoubleClick Bid Manager API documentation.

Our Google Display & Video 360 connector is now generally available.

Read our Google Display & Video 360 connector documentation.

Google Drive

Our Google Drive connector is now generally available.

Read our Google Drive connector documentation.

Jira

Now, we store a separate cursor for each project and use it to track updates in project issues. We can now sync all issues regardless of when the connecting user in your Jira instance was granted permissions for the relevant projects. Previously, we had a single cursor for all issues. When the user was granted permissions for a project after the initial sync had completed, we were not able to capture the old issues created before the user was granted permissions.

We now fetch issues in parallel threads, one for each project, which improves the extract time of the connector. We are gradually migrating all existing connectors to this new mode.

Outreach

Our Outreach connector is now generally available.

Read our Outreach connector documentation.

Salesforce

We now convert empty strings ("") to null objects. We are gradually rolling out this new feature to all existing connectors. If you'd like to enable this feature on your connector, contact our support team.

Typeform

We now sync the free-text response of the Other option in multiple choice questions to the text column of the RESPONSE_ANSWER table.

Workday RaaS

We now automatically postpone your syncs by six hours if they fail due to Workday's weekly maintenance activities. Workday performs maintenance weekly, monthly, and quarterly during a designated period, during which it is unavailable for use. After the maintenance period, your connector resumes the sync on the scheduled sync frequency.

Zoho CRM

We now support custom fields for the standard modules of Zoho CRM. Previously, we only supported custom fields for custom modules. For more information about custom fields, see Working with Custom Fields.

Our Zoho CRM connector is now generally available.

Read our Zoho CRM connector documentation.

REST API Improvements

Amazon S3

You can now use a custom external_id parameter when you create a new Amazon S3 connector with the Fivetran REST API.

NOTE: Connectors created in the Fivetran dashboard do not support custom external_id values, so they use the connector's group_id by default instead.