June 2020

Schema changes

AdRoll

We have added a new field, advertisable_name, to the following tables:

• ALL_AUDIENCE
• ALL_CAMPAIGNS
• ALL_ADS
• ALL_ADGROUPS

Fivetran Platform Connector

We have added a new table, CREDITS_USED. This table stores the monthly credit usage summary per destination.

Jira

We have added a new way to sync issue field values and their historical changes. Previously, users whose ISSUE table had a large number of fields found that it was generating an overwhelming amount of ISSUE_[FIELD_NAME]_HISTORY tables.

Now, the ISSUE table contains only columns for the Jira standard fields.

We added a FIELD table that contains information about all fields. We sync all values from the custom non-array fields and their history items to a single ISSUE_FIELD_HISTORY table. We sync all values from the array fields and their history items to a single ISSUE_MULTISELECT_HISTORY table. We sync initial null values from history items to the history tables to provide a better historical analysis.

If you would like to use this new method, contact our support team to enable it. Then, create a new connector or re-sync the existing one.

Mandrill

We have changed the data type of the location_longitude and location_latitude columns in the EVENT table from DOUBLE to STRING.

Microsoft Advertising

We have made multiple schema changes to support all the available fields for the reporting tables. See Microsoft’s documentation about reporting value sets.

We have added the following fields to the CAMPAIGN_PERFORMANCE_HOURLY_REPORT, CAMPAIGN_PERFORMANCE_DAILY_REPORT, AD_GROUP_PERFORMANCE_HOURLY_REPORT, AD_GROUP_PERFORMANCE_DAILY_REPORT, AD_PERFORMANCE_HOURLY_REPORT, AD_PERFORMANCE_DAILY_REPORT, KEYWORD_PERFORMANCE_HOURLY_REPORT, and KEYWORD_PERFORMANCE_DAILY_REPORT tables:

• all_conversions
• all_conversion_rate
• all_cost_per_conversion
• all_return_on_ad_spend
• all_revenue
• all_revenue_per_conversion
• view_through_conversions

We have added the following fields to the CAMPAIGN_IMPRESSION_PERFORMANCE_HOURLY_REPORT, CAMPAIGN_IMPRESSION_PERFORMANCE_DAILY_REPORT, AD_GROUP_IMPRESSION_PERFORMANCE_HOURLY_REPORT, and AD_GROUP_IMPRESSION_PERFORMANCE_DAILY_REPORT tables:

• all_conversions
• all_conversion_rate
• all_cost_per_conversion
• all_return_on_ad_spend
• all_revenue
• all_revenue_per_conversion
• view_through_conversions
• top_impression_rate_percent
• top_impression_share_percent
• top_impression_share_lost_to_budget_percent
• top_impression_share_lost_to_rank_percent
• absolute_top_impression_rate_percent
• absolute_top_impression_share_lost_to_budget_percent
• absolute_top_impression_share_lost_to_rank_percent

NetSuite SuiteAnalytics

We now add a _fivetran_synced column while syncing empty tables and columns from the source.

We have added the following new tables:

• ADDRESSES
• BILL_OF_MATERIALS
• BOM_REVISION_COMPONENTS
• BOM_REVISIONS
• CURRENCY_EXCHANGE_RATE_TYPES
• CURRENCY_EXCHANGE_RATES
• EMPLOYEE_CURRENCY_MAP
• INBOUND_SHIPMENTS
• ITEM_COLLECTION_ITEM_MAP
• ITEM_COLLECTIONS
• ITEM_PRICE_HISTORY
• LOCATION_COSTING_GROUPS
• LOCATION_COSTING_GRP_LOCATIONS
• PURCHASE_CHARGE_RULES
• RESOURCE_GROUP_ENTITY_MAP
• RESOURCE_GROUPS
• SUBSCRIPTION_LINE_REVISIONS
• TRANSACTION_HISTORY
• UNLOCKED_TIME_PERIODS

Outbrain

We have removed the following fields from the tables below:

• status from the PROMOTED_LINK_HISTORY table

• total_value from the CAMPAIGN_REPORT, GEOLOCATION_REPORT, PROMOTED_LINK_REPORT, and SECTION_REPORT tables

Recurly

We have added three new fields to replace the following fields in the TRANSACTION table:

• payment_gateway_id replaces payment_gatewayid
• payment_gateway_type replaces payment_gatewaytype
• payment_gateway_name replaces payment_gatewayname

Update your queries with the new field names to sync data into these fields.

Stripe

We have added a new table, USAGE_RECORD. This table stores the usage records of a subscription.

Zendesk Support

We have added a new column, created_at, to the SCHEDULE table.

Improvements

Adobe Analytics

We now support JSON Web Token (JWT) Authentication for the Adobe Analytics connector. All new connections created after June 29, 2020 will support JWT authentication.

Amazon S3

We now exclude S3 objects that have been archived to Glacier storage class from our data syncs. If you want us to sync these objects, restore them to standard storage.

Asana

You can now control which projects will be synced. Project IDs are now listed in the Specific Projects section of the Fivetran setup form.

Azure Synapse

We have changed the way we load data from our Azure Blob Storage staging area into Azure Synapse. We now use COPY instead of PolyBase.

Using COPY instead of PolyBase improves data security. During a sync, the destination is now provided with limited and temporary access to only the files that need to be synced instead of the entire storage account. We are gradually rolling out this change to all existing connections.

Braintree

We have improved the sync speed for our Braintree connector by changing the sync strategy of the TRANSACTION table. We now use multiple API calls with filters to sync data from the TRANSACTION table.

Google Ad Manager

We have upgraded our Google Ad Manager connector from version 201908 to 201911 of the Ad Manager API and handled deprecations and updates. Read more about the deprecations and changes in Google's Ad Manager API documentation.

• New dimensions:

  • cms_metadata
  • child_network_id
  • child_network_code

• Deprecated dimensions:

  • proposal_line_item_id
  • proposal_line_item_name
  • proposal_id
  • proposal_name
  • all_salespeople_id
  • all_salespeople_name
  • sales_team_id
  • sales_team_name
  • proposal_agency_id
  • proposal_agency_name
  • product_id
  • product_name
  • product_template_id
  • product_template_name
  • rate_card_id
  • rate_card_name
  • workflow_id
  • workflow_name
  • package_id
  • package_name
  • product_package_id
  • product_package_name

• New columns:

  • unique_reach_frequency
  • unique_reach_impressions
  • unique_reach
  • dp_active_view_eligible_impressions
  • dp_active_view_measurable_impressions
  • dp_active_view_viewable_impressions
  • dp_active_view_measurable_impressions_rate
  • dp_active_view_viewable_impressions_rate

• Deprecated columns:

  • reach_frequency
  • reach_average_revenue
  • reach
  • contracted_revenue_contracted_net_revenue
  • contracted_revenue_local_contracted_net_revenue
  • contracted_revenue_contracted_gross_revenue
  • contracted_revenue_local_contracted_gross_revenue
  • contracted_revenue_contracted_vat
  • contracted_revenue_local_contracted_vat
  • contracted_revenue_contracted_agency_commission
  • contracted_revenue_local_contracted_agency_commission
  • sales_contract_contracted_impressions
  • sales_contract_contracted_clicks
  • sales_contract_contracted_volume
  • sales_contract_budget
  • sales_contract_remaining_budget
  • sales_contract_buffered_impressions
  • sales_contract_buffered_clicks
  • scheduled_scheduled_impressions
  • scheduled_scheduled_clicks
  • scheduled_scheduled_volume
  • scheduled_scheduled_net_revenue
  • scheduled_local_scheduled_net_revenue
  • scheduled_scheduled_gross_revenue
  • scheduled_local_scheduled_gross_revenue
  • sales_total_total_budget
  • sales_total_total_remaining_budget
  • sales_total_total_contracted_volume
  • sales_total_total_contracted_net_revenue
  • sales_total_local_total_contracted_net_revenue
  • sales_total_total_contracted_gross_revenue
  • sales_total_local_total_contracted_gross_revenue
  • sales_total_total_contracted_agency_commission
  • sales_total_local_total_contracted_agency_commission
  • sales_total_total_contracted_net_revenue_with_vat
  • sales_total_local_total_contracted_net_revenue_with_vat
  • sales_total_total_scheduled_volume
  • sales_total_total_scheduled_net_revenue
  • sales_total_local_total_scheduled_net_revenue
  • sales_total_total_scheduled_gross_revenue
  • sales_total_local_total_scheduled_gross_revenue
  • unified_revenue_unreconciled_net_revenue
  • unified_revenue_local_unreconciled_net_revenue
  • unified_revenue_unreconciled_gross_revenue
  • unified_revenue_local_unreconciled_gross_revenue
  • unified_revenue_forecasted_net_revenue
  • unified_revenue_local_forecasted_net_revenue
  • unified_revenue_forecasted_gross_revenue
  • unified_revenue_local_forecasted_gross_revenue
  • unified_revenue_unified_net_revenue
  • unified_revenue_local_unified_net_revenue
  • unified_revenue_unified_gross_revenue
  • unified_revenue_local_unified_gross_revenue
  • unified_revenue_unified_agency_commission
  • unified_revenue_local_unified_agency_commission
  • unified_revenue_unreconciled_volume
  • unified_revenue_forecasted_volume
  • unified_revenue_unified_volume
  • expected_revenue_expected_net_revenue
  • expected_revenue_local_expected_net_revenue
  • expected_revenue_expected_gross_revenue
  • expected_revenue_local_expected_gross_revenue
  • sales_pipeline_pipeline_net_revenue
  • sales_pipeline_local_pipeline_net_revenue
  • sales_pipeline_pipeline_gross_revenue
  • sales_pipeline_local_pipeline_gross_revenue
  • sales_pipeline_pipeline_agency_commission
  • sales_pipeline_local_pipeline_agency_commission
  • reconciliation_dfp_volume
  • reconciliation_third_party_volume
  • reconciliation_reconciled_volume
  • reconciliation_discrepancy
  • reconciliation_reconciled_revenue
  • reconciliation_impression_discrepancy
  • reconciliation_click_discrepancy
  • reconciliation_revenue_discrepancy
  • billing_billable_net_revenue
  • billing_local_billable_net_revenue
  • billing_billable_gross_revenue
  • billing_local_billable_gross_revenue
  • billing_billable_net_revenue_before_manual_adjustment
  • billing_local_billable_net_revenue_before_manual_adjustment
  • billing_billable_gross_revenue_before_manual_adjustment
  • billing_local_billable_gross_revenue_before_manual_adjustment
  • billing_billable_vat
  • billing_local_billable_vat
  • billing_billable_agency_commission
  • billing_local_billable_agency_commission
  • billing_cap_quantity
  • billing_billable_volume
  • billing_delivery_rollover_from_previous
  • billing_realized_cpm
  • billing_realized_rate
  • discounts_breakdown_contracted_net_overall_discount
  • discounts_breakdown_billable_net_overall_discount
  • discounts_breakdown_contracted_net_non_billable

• Deprecated dimension attributes:

  • proposal_start_date_time
  • proposal_end_date_time
  • proposal_creation_date_time
  • proposal_sold_date_time
  • proposal_is_sold
  • proposal_probability_to_close
  • proposal_status_v2
  • proposal_archival_status
  • proposal_currency
  • proposal_exchange_rate
  • proposal_agency_commission_rate
  • proposal_vat_rate
  • proposal_discount
  • proposal_advertiser_discount
  • proposal_advertiser
  • proposal_advertiser_id
  • proposal_agencies
  • proposal_agency_ids
  • proposal_po_number
  • proposal_primary_salesperson
  • proposal_secondary_salespeople
  • proposal_creator
  • proposal_sales_planners
  • proposal_pricing_model
  • proposal_billing_source
  • proposal_billing_cap
  • proposal_billing_schedule
  • proposal_applied_team_names
  • proposal_approval_stage
  • proposal_inventory_release_date_time
  • proposal_local_budget
  • proposal_local_remaining_budget
  • proposal_flat_fee
  • proposal_is_programmatic
  • proposal_line_item_start_date_time
  • proposal_line_item_end_date_time
  • proposal_line_item_rate_type
  • proposal_line_item_reservation_status
  • proposal_line_item_cost_per_unit
  • proposal_line_item_local_cost_per_unit
  • proposal_line_item_cost_per_unit_gross
  • proposal_line_item_local_cost_per_unit_gross
  • proposal_line_item_type_and_priority
  • proposal_line_item_size
  • proposal_line_item_archival_status
  • proposal_line_item_product_adjustment
  • proposal_line_item_buffer
  • proposal_line_item_target_rate_net
  • proposal_line_item_billing_source
  • proposal_line_item_billing_cap
  • proposal_line_item_billing_schedule
  • proposal_line_item_goal_percentage
  • proposal_line_item_cost_adjustment
  • proposal_line_item_comments
  • proposal_line_item_reconciliation_status
  • proposal_line_item_last_reconciliation_date_time
  • proposal_line_item_flat_fee
  • product_package_item_archival_status
  • proposal_line_item_type
  • product_template_rate_type
  • product_template_status
  • product_template_type_and_priority
  • product_template_product_type
  • product_template_description
  • product_product_template_name
  • product_rate_type
  • product_status
  • product_type_and_priority
  • product_product_type
  • product_notes
  • product_inventory_sizes
  • product_rate
  • packaged_product_rate
  • proposal_agency_type
  • proposal_agency_credit_status
  • proposal_agency_external_id
  • proposal_agency_comment
  • salespeople_proposal_contribution
  • salesperson_proposal_contribution
  • product_package_notes
  • product_package_items
  • product_package_status
  • package_comments
  • package_start_date_time
  • package_end_date_time

Google Ads

When creating a connector, now you can select specific manager accounts (former sub-MCC accounts) to sync. Fivetran will automatically find and sync clients of these manager accounts. Read more about sync modes in our Google Ads Setup Guide.

Google Campaign Manager 360

You can now enable or disable all REACH dimension combinations in the report.

Google Sheets

We now skip the _fivetran_synced column if it is exist in the source Google Sheet. We use this name for the system column that keeps track of when each row was last successfully synced.

Greenhouse

We now sync job opening updates with increased accuracy.

Jira

We have improved the mechanism that detects the changelog. Now, if a fieldId is not present in the Jira response, we will find the changelog entries using a fieldname. This improvement prevents data integrity issues when the current field value is written into the history table as initial.

JSONL support

We now support jsonl file extensions in all file connectors.

Lever

We have fixed a bug where deleted records were incorrectly synced to your destination. Previously, deleted rows had false as their _fivetran_deleted value, but they now have true as their _fivetran_deleted value. Any previously synced deleted records will be corrected in your next sync.

Microsoft Advertising

You can now select the *_history tables you want to sync from the Schema tab on your connector details page.

NetSuite SuiteAnalytics

We now create empty tables in the destination even if the source tables are empty (contain only Null values).

We now support NetSuite 2020 Release 1 API version. We will gradually roll out this change.

PostgreSQL

We now replicate empty tables in a PostgreSQL source database as empty tables in the destination.

Salesforce

We no longer capture deletes for tables with names ending with _HISTORY, for example, OBJECT_NAME_HISTORY. Deletes from these tables contributed an unacceptably high amount of monthly active rows.

You can now select the FIVETRAN_API_CALL and FIVETRAN_QUERY tables to sync from the Schema tab on your connector details page.

Snowplow

You can now add multiple Snowplow connectors per destination.

SQL Server

We have changed our sync strategy for users who chose to exclude new tables from their data syncs by default. Our new sync method improves your sync speed, especially if you have a large number of tables.

Stripe

We can now enable support for your Stripe Connected Accounts on your connector. Contact our Support team to enable this feature on your connector.

REST API Improvements

Create a Destination Public API Endpoint Extension

We have expanded the current public API endpoint Create a Destination. Now, you can:

• control whether setup test are executed automatically by specifying the run_setup_tests parameter.
• create a Snowflake destination with password-based or key-based authentication types. Read our updated documentation about Snowflake destination config.

Google Ad Manager

You can now configure your Google Ad Manager connector using the Fivetran REST API. This feature is in BETA and available only for Standard and Enterprise accounts.

New Public API Endpoints

We have implemented new Connector, Group, and Destination Management API endpoints for our REST API. You can now leverage the following new features:

• Create a Group
• Create a Destination
• Modify a Destination
• Reload a Connector Schema Config