May 2018

Group-specific SSH keys

Fivetran now supports unique SSH keys per destination/group for SSH tunnels, SFTP, Marin, etc.

Link to log service

You can now navigate directly to your log service from the Connection Details page of your Fivetran dashboard.

Logs

Logs support

We have added logs for the following applications:

• AdRoll
• Amazon S3
• Apple Search Ads
• Amazon CloudFront
• Dropbox
• Eloqua
• Email
• Freshdesk
• FTP
• Google Cloud Storage
• HelpScout
• Intercom
• Klaviyo
• Mailchimp
• Marin Software
• Azure Blob Storage
• Microsoft Advertising
• Mixpanel
• MySQL
• Pardot
• Recurly
• SendGrid
• SFTP
• Verizon Media
• Zuora

Okta SAML support

You can now use Okta to log in to Fivetran. To add Fivetran to Okta and configure SAML, follow these instructions.

We also support Just In Time (JIT) provisioning, meaning that we'll automatically create an account for each user the first time they log in with Okta.

New connectors

Azure SQL Database

Azure SQL Database is a managed relational cloud database service that you can use to build data-driven applications and websites in the programming language of your choice, without needing to manage infrastructure.

See our Azure SQL Database setup instructions.

Google Search Console

Google Search Console allows you to monitor, maintain, and troubleshoot your site's presence in Google Search results.

Read our Google Search Console documentation.

MySQL on Google Cloud SQL

MySQL on Google Cloud SQL is a fully-managed database service that makes it easy to set up, maintain, manage, and administer your MySQL relational databases on Google Cloud Platform.

See our Google Cloud SQL for MySQL setup instructions.

PostgreSQL on Azure Database

PostgreSQL on Azure Database is a managed relational cloud database service that can handle mission-critical workloads with predictable performance, security, high availability, and dynamic scalability.

See our Azure Database for PostgreSQL setup instructions.

PostgreSQL on Google Cloud SQL Database

PostgreSQL on Google Cloud SQL Database is a fully-managed database service that makes it easy to set up, maintain, manage, and administer your PostgreSQL relational databases on Google Cloud Platform.

See our Google Cloud PostgreSQL setup instructions.

Sigma Computing

Sigma Computing is the next-generation of analytics for cloud data warehouses with a familiar spreadsheet-like interface that gives business experts the power to ask any question of their data no matter the query.

See our Sigma Computing setup instructions.

Schema changes

Eloqua

We have added two new tables, CONTACT_ACTIVITY and VISITOR.

• The CONTACT_ACTIVITY table contains contact activity data.
• The VISITOR table contains details about visitors to your website.

GitHub

We have added a new field, archived, to the REPOSITORY table.

HubSpot

We have added two new fields, callee_object_type and callee_object_id, to the ENGAGEMENT_CALL table.

Intercom

We have added three new columns to the COMPANY table:

• size
• website
• industry

Jira

We have dropped the primary key constraint from the value column in the ISSUE_[FIELD]_HISTORY and ISSUE_[FIELD] tables.

Kantata

Kantata's custom field value type currency contains both cents and currency code data (for example, a currency value of [12,USD] = 12 cents USD). We have split currency data into two columns, <custom_field_name>_amount_in_cents and <custom_field_name>_currency_code.

Mandrill

We have replaced id with _fivetran_id as the only primary key column in the ACTIVITY and ACTIVITY_TAG tables.

Marin Software

We have converted the settings tables to be history tables by making the last_modified field a second primary key. We have renamed the following tables to reflect this change:

We have separated the TRACKING_VALUE and DIMENSION tables into various child tables, which each have a foreign key relationship with their respective parent entity.

NetSuite SuiteAnalytics

We no longer sync the SUBSCRIPTION_CO_LINES table.

Recurly

We have added two new columns, start_date and end_date, to the ADJUSTMENT table.

SendGrid

We have renamed the CAMPAIGN_CATEGORY table to CATEGORY. It has a foreign key relationship to the EVENT table.

We have made the following changes to the EVENT table:

• The primary key is now _fivetran_id.
• We have renamed the id field to sg_event_id.

Stripe

We have added a new table, FILE_UPLOAD.

We have added the following new columns to the tables below:

Improvements

Amazon S3

Our connector setup form now correctly handles directory patterns that don't end with /.

Azure Blob Storage

Azure recommends using Parquet files instead of CSVs, because they don't interpret CSVs reliably. Fivetran will write your data to a Parquet file and upload it to Azure Blob Storage.

Azure SQL Database

We have made the following improvements to our Azure SQL Database connector:

• We now use case-sensitive collation for STRING columns so that JOINs handle case correctly.

• We now create column store indexes for all tables in Azure SQL Database. If column store indexing fails, you will be automatically notified that you need to upgrade the resource class of the user.

Email

When we encounter malformed files during our syncs, we now skip them and send you a warning so that you can fix the files in question. Previously, our entire sync would fail if malformed files were present.

Files

We now parse ISO DATE format (yyyy-mm-dd-hh:mm:ss) correctly.

Functions

We have added two new response root nodes, delete and schema:

• delete contains the entities and records to be deleted. Fivetran does not delete these records; instead, we mark them as deleted by setting the _fivetran_deleted column value to true.

• schema specifies primary key columns for each entity.

If you are specifying the delete node, you must also specify schema node.

For more information, see our Functions documentation.

Google Ad Manager

We have reduced the data window dynamically by one day in case of report failure.

Google Campaign Manager 360

In the connector setup form, you can now select only one Floodlight Configuration ID per account.

Kantata

Kantata returns choice ID data for custom field types single and multi. We now fetch the choice value instead of the choice ID.

Mandrill

Mandrill silently revises aggregate clicks and opens metrics in the API results, so we have implemented a 30-day rollback sync to accurately capture this data.

MySQL

We now fully support the SET data type and consistently sync it as a JSON array. Previously, we synced SET as STRING during the initial import and as INT when processed from the binary logs. If any values were incorrectly synced prior to this change, you will need to initiate a re-sync to correct them.

NetSuite SuiteAnalytics

We have updated the JDBC driver to version 7.20.50.

PostgreSQL

For logical replication mode, we have switched to a different logical decoding plugin that consumes significantly less disk and memory resources on the database instance.

We can now correctly parse PostgreSQL array data types with escaped characters.

Recharge

We have corrected the way our system interprets timestamps. The Recharge API produces timestamps that look like 2018-01-01T00:00:00. Previously, we interpreted these timestamps as UTC, when they were actually Eastern Time (ET). This meant that all destination timestamps were off by 4 or 5 hours, depending on the daylight saving time changes. All timestamps are now correctly converted from ET to UTC before they arrive in your destination. Note that there is still one hour each year where the timestamp is ambiguous due to daylight saving time. Recharge is working on a fix for this in their API.

SQL Server

We can now sync columns with the DATETIMEOFFSET data type. The values are converted to UTC using the timezone information present in the data.

Zendesk Support

Zendesk uses various non-standard time formats in different regions. We can now parse non-standard times like Mon Jan 1 2018, 00:00:00 GMT-0300 (Hora oficial do Brasil) and Mon Apr 30 2018 15:44:04 GMT+0000 when they come from Zendesk.