Webhooks
Our Webhook API is a RESTful API accessed by using either an HTTP client, such as wget or curl, or an HTTP library, which is a part of most modern programming language packages.
Our webhooks test project is a simple example that demonstrates how to ingest webhooks from Fivetran.
Create webhooks
Learn how to create and manage webhooks for Fivetran events.
TIP: For examples of how to set up and use webhooks, see the code sample for your environment:
Creating a webhook is a two-step process:
- Set up your application to receive webhook payloads at a secure, public URL.
- Create a webhook with Fivetran to fire in response to particular events.
The Fivetran Webhook REST API lets you manage webhooks. You can also use the REST API to change the configuration of your webhook. For example, you can modify the payload URL and the list of events.
URL
The payload URL must be a secure site (HTTPS). If an insecure (HTTP) payload URL is received, Fivetran returns an HTTP 400 Bad Request
error. During the webhook creation or activation, we perform a single POST
request to the specified URL with a testing payload. If endpoint is not reachable or returns an unsuccessful status code, Fivetran does not create the webhook and returns an HTTP 400 Bad Request
error. You can skip this check by setting active
to false
during the webhook creation, but we still perform this test during the webhook activation.
Active
You can disable the delivery of webhook payloads by setting active
to false
.
Events
When configuring a webhook, you can use the API to choose which events send you payloads. Subscribing to only specific events you want to listen for minimizes the number of HTTPS requests to your server.
We support the following events:
- sync_start
- sync_end
- status (deprecated)
- dbt_run_start
- dbt_run_succeeded
- dbt_run_failed
- connection_successful
- connection_failure
- create_connector
- pause_connector
- resume_connector
- edit_connector
- delete_connector
- force_update_connector
- resync_connector
- resync_table
NOTE: Fill out our survey if you want more event types supported by our Webhook Management API.
Retries
Sometimes, a webhook that we sent for a particular event instance (for example, a transformation's start or a connector's sync end) may fail for various reasons, such as the relevant endpoint being unavailable, etc. In that case, we retry the webhook for up to 24 hours in increasing time intervals. We stop sending the webhook for the given event instance after retrying for 24 hours.
We use an exponential algorithm to calculate the intervals between the retries. We wait 6 minutes before the first retry, 21 minutes before the second retry, and so on. This approach allows us to make up to five webhook retries within the 24-hour period.
See how Fivetran calculates the times of retries following a failed initial attempt in the following example:
Initial attempt and following retries | Time |
---|---|
Initial attempt | 10:00 |
1st retry | 10:06 |
2nd retry | 10:27 |
3rd retry | 11:45 |
4th retry | 16:25 |
5th retry | 9:13 the next day |
NOTE: For each webhook event type within a given context (a given connector/transformation in a given group in your account), we send webhooks for each event instance independently of the other instances of that event. For example, you created a sync start webhook for a connector with the sync frequency of 6 hours. At the sync start at 00:12, Fivetran sent a sync start webhook that failed. Regardless of the number of retries and the final status of this webhook, Fivetran still will send another sync start webhook at the next sync start at 6:12.
Connection request timeout
Webhooks have a 10-second connection request timeout. If your endpoint doesn't return an HTTP 200 OK
status in 10 seconds, the webhook request fails, and Fivetran retries it based on the schedule outlined in Retries.
Automatic deactivation
If a webhook consistently fails for more than three days and only responds with non-2xx codes, we deactivate it.
To activate the webhook again, ensure that the setup is correct by retrieving the webhook details and updating them with corrected information along with passing "active": true
in the request. Alternatively, you can create a new webhook.
Signing
The webhook payload can be signed if you specified a secret during the webhook creation. The signature is calculated based on the payload body using the SHA-256 HMAC algorithm.
Example of webhook payload
Fivetran sends webhooks with the following payload structure:
{
"event" : "sync_end",
"created" : "2021-08-18T11:38:34.386Z",
"connector_type" : "asana",
"connector_id" : "mystified_presiding",
"connector_name" : "mystified_presiding",
"sync_id" : "mystified_presiding",
"destination_group_id" : "deck_enjoy",
"data" : {
"status" : "SUCCESSFUL"
}
}
Fields
Name | Description | Possible Values |
---|---|---|
event | The event type | |
created | The event generation date | |
connector_type | The type of the connector for which the webhooks is sent | |
connector_id | The ID of the connector for which the webhooks is sent | |
connector_name | The name of the connector for which the webhooks is sent | |
sync_id | The sync for which the webhooks is sent | |
destination_group_id | The destination group ID of the connector for which the webhooks is sent | |
data | The response payload object. The object fields vary depending on the event type. | |
status | The webhook creation status | SUCCESSFUL , FAILED |