Google Cloud Storage API Configuration

Request

POST https://api.fivetran.com/v1/connections

{
  "group_id": "group_id",
  "service": "gcs",
  "trust_certificates": true,
  "trust_fingerprints": true,
  "run_setup_tests": true,
  "paused": false,
  "pause_after_trial": false,
  "sync_frequency": 1440,
  "data_delay_sensitivity": "NORMAL",
  "data_delay_threshold": 0,
  "daily_sync_time": "14:00",
  "schedule_type": "auto",
  "connect_card_config": {
    "redirect_uri": "https://your.site/path",
    "hide_setup_guide": true
  },
  "proxy_agent_id": "proxy_agent_id",
  "private_link_id": "private_link_id",
  "networking_method": "Directly",
  "hybrid_deployment_agent_id": "hybrid_deployment_agent_id",
  "config": {
    "quote_character_enabled": true,
    "prefix": "folder_path",
    "non_standard_escape_char": true,
    "pattern": "file_pattern",
    "json_delivery_mode": "Packed | Unpacked",
    "skip_after": 0,
    "quote_char": "string",
    "secret_key": "secret_key",
    "empty_header": true,
    "use_pgp_encryption_options": true,
    "skip_before": 0,
    "signer_public_key": "<signer-public-key-used>",
    "delimiter": "string",
    "file_type": "csv | log | json | jsonl | infer | tsv | avro | parquet | xml | spreadsheet | unstructured",
    "on_error": "fail | skip",
    "archive_pattern": "regex_pattern",
    "pgp_secret_key": "<secret-key-used>",
    "auth_type": "FIVETRAN_SERVICE_ACCOUNT | CUSTOM_SERVICE_ACCOUNT",
    "append_file_option": "upsert_file | append_file | upsert_file_with_primary_keys",
    "escape_char_options": "CUSTOM_ESCAPE_CHAR | DELIMITED_ONLY",
    "escape_char": "string",
    "bucket": "bucket_name",
    "line_separator": ";",
    "null_sequence": "string",
    "files": [
      {
        "file_pattern": "string",
        "table_name": "string"
      }
    ],
    "compression": "bz2 | gz | gzip | zip | tar | tar_bz2 | tar_gz | uncompressed | infer",
    "pgp_pass_phrase": "<pass-phrase-used>",
    "schema": "schema_name",
    "table_group_name": "string"
  }
}

Config parameters

Name	Description
`quote_character_enabled`	If you don’t want to use an enclosing character while processing the files, set the value to false. By default, the value is set to true and is considered " as quote character.
`prefix`	All files and folders under this folder path will be searched for files to sync.
`non_standard_escape_char`	Use this if your CSV generator uses non-standard ways of escaping characters. Default value: `false`.
`pattern`	All files in your search path matching this regular expression will be synced. We have discontinued this field, and will delete it by August 31, 2025.
`json_delivery_mode`	Control how your JSON data is delivered into your destination
`skip_after`	We will skip over the number of lines specified at the end so as to not introduce aberrant data into your destination.
`quote_char`	You can specify the enclosing character used in your CSVs here. Fivetran will consider " as the quote character if this field is empty and quote_character_enabled is set to true.
`secret_key`	Your JSON Private Key. Used to authorize service account. Required if you use a Custom Service Account to authenticate the storage bucket.
`empty_header`	Optional. If your CSV generating software doesn't provide header line for the documents, Fivetran can generate the generic column names and sync data rows with them.
`use_pgp_encryption_options`	Set to `true` if files are encrypted using PGP in the GCS bucket. Default value: `false`.
`skip_before`	We will skip over the number of lines specified before syncing data.
`signer_public_key`	The contents of the signer's public key file. Must be populated if `use_pgp_encryption_options` is set to `true` and PGP encrypted files are signed.
`delimiter`	You can specify your the delimiter that your CSVs use here. Fivetran generally tries to infer the delimiter, but in some cases this is impossible.
`file_type`	If your files are saved with improper extensions, you can force them to by synced as the selected file type.
`on_error`	If you know that your files contain some errors, you can choose to have poorly formatted lines skipped. We recommend leaving the value as fail unless you are certain that you have undesirable, malformed data.
`archive_pattern`	Files inside of compressed archives with filenames matching this regular expression will be synced.
`pgp_secret_key`	The contents of your PGP secret key file. Must be populated if `use_pgp_encryption_options` is set to `true`.
`auth_type`	Authorization type. Required for storage bucket authentication.
`append_file_option`	If you know that the source completely over-writes the same file with new data, you can append the changes instead of upserting based on filename and line number.
`escape_char_options`	Approach used by CSV parser. Default value: `CUSTOM_ESCAPE_CHAR`. required for CSV parsing when `non_standard_escape_char` is `true`.
`escape_char`	If your CSV generator follows non-standard rules for escaping quotation marks, you can set the escape character here.
`bucket`	The name of the GCS bucket.
`line_separator`	You can specify the custom line separator for your CSV files. The line separator is used in files to separate one row from the next.
`null_sequence`	If your CSVs use a special value indicating null, you can specify it here.
`files`	Mapped file name patterns to a destination table.
`files[0].file_pattern`	All files in your search path matching this regular expression will be synced per table.
`files[0].table_name`	All files matching the above pattern will be synced to this table.
`compression`	The secrets that should be passed to the function at runtime.
`pgp_pass_phrase`	The PGP passphrase used to create the key. Must be populated if `use_pgp_encryption_options` is set to `true`.
`schema`	Destination schema name. Schema name is permanent and cannot be changed after connection creation
`table_group_name`	Table group name

Authorization

There are two ways to authorize this connector type:

By specifying the values for the bucket, auth_type and secret_key parameters in the request.

POST https://api.fivetran.com/v1/connections

{
  "service": "gcs",
  "group_id": "group_id",
  "config": {
    "bucket": "bucket_name",
    "auth_type": "FIVETRAN_SERVICE_ACCOUNT | CUSTOM_SERVICE_ACCOUNT",
    "secret_key": "secret_key"
  }
}

By using the Connect Card or the Fivetran dashboard.