Fivetran External Secret Management with HashiCorp Vault Setup Guide

Follow our setup guide to connect your HashiCorp Vault to Fivetran to use it as an External Secret Manager.

Prerequisites

To connect your HashiCorp Vault to Fivetran, you need the following:

A Fivetran account with the Account Administrator role
A running HashiCorp Vault instance accessible from the internet
(Only if using AWS IAM authentication) An AWS account with permissions to manage IAM roles
(Only for Hybrid Deployment using AWS IAM authentication) A Hybrid Deployment agent running on an AWS EC2 instance
(Only for Hybrid Deployment using Kubernetes and JWT authentication) A Hybrid Deployment agent running on Kubernetes, with the OIDC discovery URL of your Kubernetes cluster

Setup instructions

Follow the instructions for the deployment model and authentication method of your choice:

Get the Account ID from Fivetran

In the Fivetran dashboard, go to Account Settings > General > External Secrets Managers.
Click Create new secrets manager.
Select SaaS Deployment and HashiCorp Vault.
Make a note of the pre-populated Account ID value shown in the form. You will need it when configuring the AWS auth method in Vault. Do not close this form.

Create an IAM role in AWS

Log in to the AWS console and go to IAM > Roles.
Click Create role.
Select AWS account as the trusted entity type.
Select Another account and enter Fivetran's AWS account ID: 834469178297.
You do not need to assign any policies. Click Next again.
Enter a Role name and click Create role.
Click on the newly created role and verify that the trust policy has sts:AssumeRole as the action on the 834469178297 principal.
Make a note of the role ARN. You will need it to configure both Vault and Fivetran.

Configure HashiCorp Vault

Enable the AWS auth method:
```
vault auth enable aws
```
Configure the AWS auth method. Replace <account_id> with the account ID you noted in Fivetran:
```
vault write auth/aws/config/client \
  iam_server_id_header_value=<account_id>
```
Create a policy that grants read access to your secrets. Replace <vault-path> with the path prefix where your secrets are stored (for example, myapp):
```
vault policy write fivetran-secrets-read - <<EOF
path "secret/data/<vault-path>" {
  capabilities = ["read"]
}
EOF
```
Create a Vault role bound to the IAM role you created in AWS. Replace <vault-role-name> with a name of your choice and <role-arn> with the ARN from substep 8 of the Create an IAM role in AWS step above:
```
vault write auth/aws/role/<vault-role-name> \
  auth_type=iam \
  bound_iam_principal_arn="<role-arn>" \
  policies=fivetran-secrets-read \
  ttl=1h \
  max_ttl=4h
```
Make a note of the <vault-role-name>. You will need it to configure Fivetran.

Configure External Secret Manager in Fivetran

Return to the External Secret Manager creation form in Fivetran.
Provide the following:
- Secret Manager Name: An internal name for this secrets manager in Fivetran.
- Vault Address: Your Vault server's address.
- Vault Role: The Vault role name you created.
- Vault Path: The path at which your secrets are stored (for example, myapp/config).
- Namespace (optional): Your Vault namespace, if you have one. Leave empty otherwise.
- Auth Method: Select AWS IAM. In the AWS Role ARN field that appears, enter the IAM role ARN you noted in step 8.
Click Add secrets manager.

Configure HashiCorp Vault

Enable the JWT auth method:
```
vault auth enable jwt
```

Configure the JWT auth method with our OIDC discovery URL:

vault write auth/jwt/config \
  oidc_discovery_url="https://container.googleapis.com/v1/projects/fivetran-donkeys/locations/us-east4/clusters/frontend-production-us-east4" \
  oidc_discovery_ca_pem=""

Create a policy that grants read access to your secrets. Replace <vault-path> with the path prefix where your secrets are stored (for example, myapp):
```
vault policy write fivetran-secrets-read - <<EOF
path "secret/data/<vault-path>" {
  capabilities = ["read"]
}
EOF
```

Create a Vault role bound to our service accounts in the orchestrator namespace. Replace <vault-role-name> with a name of your choice:

vault write auth/jwt/role/<vault-role-name> \
  role_type=jwt \
  bound_audiences="<vault-role-name>" \
  user_claim="sub" \
  bound_claims_type="glob" \
  bound_claims='{"sub": "system:serviceaccount:orchestrator:*"}' \
  policies=fivetran-secrets-read \
  ttl=1h \
  max_ttl=4h

Make a note of the <vault-role-name>. You will need it to configure Fivetran.

Configure External Secret Manager in Fivetran

In the Fivetran dashboard, go to Account Settings > General > External Secrets Managers.
Click Create new secrets manager.
Select SaaS Deployment and HashiCorp Vault.
Provide the following:
- Secret Manager Name: An internal name for this secrets manager in Fivetran.
- Vault Address: Your Vault server's address.
- Vault Role: The Vault role name you created.
- Vault Path: The path at which your secrets are stored (for example, myapp/config).
- Namespace (optional): Your Vault namespace, if you have one. Leave empty otherwise.
- Auth Method: Select JWT.
Click Add secrets manager.

This guide assumes your Hybrid Deployment agent is running on an AWS EC2 instance.

Get the Account ID from Fivetran

In the Fivetran dashboard, go to Account Settings > General > External Secrets Managers.
Click Create new secrets manager.
Select Hybrid Deployment and HashiCorp Vault.
Make a note of the pre-populated Account ID value shown in the form. You will need it when configuring the AWS auth method in Vault. Do not close this form.

Create an IAM role in AWS and attach it to your Hybrid Deployment agent

Log in to the AWS console and go to IAM > Roles.
Click Create role.
Select AWS service as the trusted entity type.
Select EC2 for Service or use case and make sure the use case is set to EC2. Click Next.
You do not need to assign any policies. Click Next again.
Enter a Role name and click Create role.
Click on the newly created role and verify that the trust policy has sts:AssumeRole as the action on the ec2.amazonaws.com principal.
Make a note of the role ARN. You will need it to configure Vault.
In the AWS console, go to EC2 > Instances.
Find and select the instance running your Hybrid Deployment agent.
Click Actions > Security > Modify IAM role.
Select the role you created and click Update IAM role.

Configure HashiCorp Vault

Enable the AWS auth method:
```
vault auth enable aws
```
Configure the AWS auth method. Replace <account_id> with the account ID you noted in Fivetran:
```
vault write auth/aws/config/client \
  iam_server_id_header_value=<account_id>
```
Create a policy that grants read access to your secrets. Replace <vault-path> with the path prefix where your secrets are stored (for example, myapp):
```
vault policy write fivetran-secrets-read - <<EOF
path "secret/data/<vault-path>" {
  capabilities = ["read"]
}
EOF
```
Create a Vault role bound to the IAM role attached to your Hybrid Deployment agent. Replace <vault-role-name> with a name of your choice and <role-arn> with the ARN from substep 8 of the Create an IAM role in AWS and attach it to your Hybrid Deployment agent step above:
```
vault write auth/aws/role/<vault-role-name> \
  auth_type=iam \
  bound_iam_principal_arn="<role-arn>" \
  policies=fivetran-secrets-read \
  ttl=1h \
  max_ttl=4h
```
Make a note of the <vault-role-name>. You will need it to configure Fivetran.

Configure External Secret Manager in Fivetran

Return to the External Secret Manager creation form in Fivetran.
Provide the following:
- Secret Manager Name: An internal name for this secrets manager in Fivetran.
- Vault Address: Your Vault server's address.
- Vault Role: The Vault role name you created.
- Vault Path: The path at which your secrets are stored (for example, myapp/config).
- Namespace (optional): Your Vault namespace, if you have one. Leave empty otherwise.
- Auth Method: Select AWS IAM with EC2 instances.
Click Add secrets manager.

Get the OIDC discovery URL of your Kubernetes cluster

Retrieve the OIDC discovery URL for your Kubernetes cluster. The format depends on your provider:

Google Kubernetes Engine (GKE): https://container.googleapis.com/v1/projects/<project>/locations/<region>/clusters/<cluster>
Amazon Elastic Kubernetes Service (EKS): Available in the EKS console under your cluster's Overview tab, labeled OpenID Connect provider URL.
Azure Kubernetes Service (AKS): https://<region>.oic.prod-aks.azure.com/<tenant-id>/<cluster-uuid>/

Configure HashiCorp Vault

Enable the JWT auth method:
```
vault auth enable jwt
```
Configure the JWT auth method with your Kubernetes cluster's OIDC discovery URL. Replace <oidc-discovery-url> with the URL you retrieved:
```
vault write auth/jwt/config \
  oidc_discovery_url="<oidc-discovery-url>" \
  oidc_discovery_ca_pem=""
```
Create a policy that grants read access to your secrets. Replace <vault-path> with the path prefix where your secrets are stored (for example, myapp):
```
vault policy write fivetran-secrets-read - <<EOF
path "secret/data/<vault-path>" {
  capabilities = ["read"]
}
EOF
```

Create a Vault role bound to service accounts in your Hybrid Deployment agent's namespace. Replace <vault-role-name> with a name of your choice and <namespace> with the Kubernetes namespace where your Hybrid Deployment agent is running:

vault write auth/jwt/role/<vault-role-name> \
  role_type=jwt \
  bound_audiences="esm-hashicorp-vault" \
  user_claim="sub" \
  bound_claims_type="glob" \
  bound_claims='{"sub": "system:serviceaccount:<namespace>:*"}' \
  policies=fivetran-secrets-read \
  ttl=1h \
  max_ttl=4h

Make a note of the <vault-role-name>. You will need it to configure Fivetran.

Enable JWT authentication in the Hybrid Deployment agent

Open the Hybrid Deployment agent configuration:

kubectl edit configmap hd-agent-config -n <namespace>

Add the following line under the data section:
```
enable_hashicorp_vault_jwt_auth: "true"
```
Save and close the file.

Restart the Hybrid Deployment agent to apply the change:

kubectl rollout restart deployment/hd-agent -n <namespace>

Configure External Secret Manager in Fivetran

In the Fivetran dashboard, go to Account Settings > General > External Secrets Managers.
Click Create new secrets manager.
Select Hybrid Deployment and HashiCorp Vault.
Provide the following:
- Secret Manager Name: An internal name for this secrets manager in Fivetran.
- Vault Address: Your Vault server's address.
- Vault Role: The Vault role name you created.
- Vault Path: The path at which your secrets are stored (for example, myapp/config).
- Namespace (optional): Your Vault namespace, if you have one. Leave empty otherwise.
- Auth Method: Select JWT.
Click Add secrets manager.

Creating secrets in HashiCorp Vault

Once your External Secret Manager is configured, add your secrets to Vault as key-value pairs under the configured Vault path. All secrets for Fivetran connectors and destinations must be stored under that same path.

Fivetran supports a basic key-value secret model only. JSON-like secrets are unpacked as key-value pairs. Store each credential as a separate key under the configured Vault path.

For example, to store credentials for multiple connectors under the same path, run:

vault kv put secret/<vault-path> \
  sql_server_password=mysecretpassword \
  sql_server_username=myuser \
  workday_username=myworkdayusername \
  workday_password=myworkdaypassword

When setting up a connection or destination that uses ESM, enter the secret key (for example, sql_server_password) in the corresponding ESM key field in Fivetran — not the secret value itself.