AWS MSK API Configuration

Request

POST https://api.fivetran.com/v1/connections

{
  "group_id": "group_id",
  "service": "aws_msk",
  "trust_certificates": true,
  "trust_fingerprints": true,
  "run_setup_tests": true,
  "paused": false,
  "pause_after_trial": false,
  "sync_frequency": 1440,
  "data_delay_sensitivity": "NORMAL",
  "data_delay_threshold": 0,
  "daily_sync_time": "14:00",
  "schedule_type": "auto",
  "connect_card_config": {
    "redirect_uri": "https://your.site/path",
    "hide_setup_guide": true
  },
  "proxy_agent_id": "proxy_agent_id",
  "private_link_id": "private_link_id",
  "networking_method": "Directly",
  "hybrid_deployment_agent_id": "hybrid_deployment_agent_id",
  "config": {
    "consumer_group": "consumer_group",
    "sync_type": "Unpacked | Packed",
    "sasl_scram512_secret": "sasl_secret",
    "schema_registry_credentials_source": "string",
    "message_type": "Avro | Json | Text | Protobuf",
    "sasl_scram512_key": "sasl_key",
    "connection_method": "direct | privatelink",
    "keystore_password": "keystore_password",
    "servers": [
      "string"
    ],
    "role_arn": "arn::your_role_arn",
    "sasl_mechanism": "SCRAM_SHA_512 | IAM",
    "security_protocol": "PLAINTEXT | TLS | SASL",
    "schema_registry_secret": "string",
    "truststore": "test_truststore",
    "keystore": "test_keystore",
    "key_password": "key_password",
    "schema_registry_urls": [
      "string"
    ],
    "schema_registry_key": "string",
    "schema": "schema_name"
  }
}

Config parameters

Name	Description
`consumer_group`	The name of consumer group created for Fivetran.
`sync_type`	The sync type. Unpacked messages must be valid JSON.
`sasl_scram512_secret`	If `sasl_mechanism` is set to `SCRAM_SHA_512`, enter your secret's `saslScram512Key`.
`schema_registry_credentials_source`	Schema Registry Credentials source
`message_type`	The Message type.
`sasl_scram512_key`	If `sasl_mechanism` is set to `SCRAM_SHA_512`, enter your secret's `saslScram512Key`.
`connection_method`	How Fivetran connects to your message brokers in the cluster
`keystore_password`	If `security_protocol` is set to `TLS`, enter your `Keystore Password`.
`servers`	Comma-separated list of Kafka servers in the `server:port` format.
`role_arn`	If `sasl_mechanism` is set to `IAM`, enter your Role ARN
`sasl_mechanism`	If `security_protocol` is set to `SASL`, enter the SASL Mechanism
`security_protocol`	The security protocol for Kafka interaction.
`schema_registry_secret`	Schema Registry Secret
`truststore`	If `security_protocol` is set to `TLS`, add the `Truststore File` as Base64 encoded string.
`keystore`	If `security_protocol` is set to `TLS`, add the `Keystore File` as Base64 encoded string.
`key_password`	If `security_protocol` is set to `TLS`, enter your `Key Password`.
`schema_registry_urls`	Your schema registry URLs
`schema_registry_key`	Schema Registry Key
`schema`	Destination schema name. Schema name is permanent and cannot be changed after connection creation

Authorization

There are four ways to authorize this connector type:

TLS: By specifying the security_protocol as "TLS" and the values for the truststore, keystore, keystore_password and key_password parameters in the request.

POST https://api.fivetran.com/v1/connections

{
  "service": "aws_msk",
  "group_id": "group_id",
  "config": {
    "truststore": "test_truststore",
    "keystore": "test_keystore",
    "keystore_password": "keystore_password",
    "key_password": "key_password",
    "security_protocol": "TLS"
  }
}

SASL - SCRAM: By specifying the security_protocol as "SASL" and the sasl_mechanism as "SCRAM_SHA_512" and the values for the sasl_scram512_key and sasl_scram512_secret parameters in the request.

POST https://api.fivetran.com/v1/connections

{
  "service": "aws_msk",
  "group_id": "group_id",
  "config": {
    "sasl_scram512_key": "sasl_key",
    "sasl_scram512_secret": "sasl_secret",
    "security_protocol": "SASL",
    "sasl_mechanism": "SCRAM_SHA_512"
  }
}

SASL - IAM: By specifying the security_protocol as "SASL" and the sasl_mechanism as "IAM" and the values for the sasl_scram512_key and sasl_scram512_secret parameters in the request.

POST https://api.fivetran.com/v1/connections

{
  "service": "aws_msk",
  "group_id": "group_id",
  "config": {
    "sasl_scram512_key": "sasl_key",
    "sasl_scram512_secret": "sasl_secret",
    "security_protocol": "SASL",
    "sasl_mechanism": "IAM"
  }
}

By using the Connect Card or the Fivetran dashboard.