Azure Monitor Setup Guide

Prerequisites

To connect Azure Monitor to Fivetran using the legacy Data Collector API, you need an Azure account with a Log Analytics workspace.

Locate workspace ID and primary key

1. Log in to Azure Portal.

2. Go to Home > Log Analytics Workspace and select your workspace from the list.

   

3. In the left navigation panel, click Agents, choose the server type, and open the Log Analytics agent instructions section.

   

4. Make a note of the Workspace ID and Primary key you see in the Log Analytics agent instructions section. You will need them to configure Fivetran.

az account set --subscription "<SubscriptionId>"

az monitor log-analytics workspace get-shared-keys \
  --resource-group "<ResourceGroup>" \
  --workspace-name "<WorkspaceName>" \
  --query "{primaryKey: primarySharedKey}"

Finish Fivetran configuration

1. Go to the external log setup form.
2. Enter the Workspace ID and Primary key you found in Step 1.
3. Click Save & Test. Fivetran will take it from here and sync your logs to Azure Monitor.

Setup test

Fivetran performs the Connect to Azure Monitor Logs setup test to check service accessibility and the credentials you provided in the setup form.

Prerequisites

To connect Azure Monitor to Fivetran using the Log Ingestion API, you need:

• An Azure account with a Log Analytics workspace.

• A registered Microsoft Entra ID application with its Client ID, Client Secret, and Directory ID.

• A configured Data Collection Endpoint (DCE) and Data Collection Rule (DCR) in your Azure environment.

• A custom table in your Log Analytics workspace configured with the following columns that Fivetran sends with each log entry:

  Column	Type	Description
  Level	string	Log level (e.g. INFO, WARNING, SEVERE)
  CreatedAt	datetime	Timestamp of the log event
  Message	string	Log message body

Locate Microsoft Entra ID application credentials

1. In the Azure Portal, navigate to Microsoft Entra ID and select App registrations.

2. Find your application in the list and open it.

3. Make a note of the Application (Client) ID and Directory (Tenant) ID shown on the overview page. You will need them to configure Fivetran.

4. If a client secret has not been generated yet, select Certificates & secrets > New client secret, fill in the description and expiry, then click Add. Make a note of the generated secret Value — it will only be shown once.

Locate the Data Collection Endpoint logs ingestion URI

1. In the Azure Portal, navigate to Monitor and select Data Collection Endpoints.

2. Select your endpoint from the list to view its properties.

3. Make a note of the Logs Ingestion URI. You will need it to configure Fivetran.

Locate the Data Collection Rule properties

1. In the Azure Portal, navigate to Monitor and select Data Collection Rules.

2. Select the required DCR from the list to open it.

3. From the Overview page of the DCR, select JSON View.

4. Make a note of the following values. You will need them to configure Fivetran:

   • immutableId - The unique identifier of the DCR.
   • logIngestion - The DCR-level ingestion endpoint under properties.endpoints, if you use the DCR's own ingestion endpoint instead of a dedicated DCE.
   • Log stream name — The name of the stream defined in the DCR dataflow that maps to your target table. It can be found in DCR JSON View dataflows[0].streams[0]

Finish Fivetran configuration

1. Go to the external log setup form.
2. Enter the Application (Client) ID, Directory (Tenant) ID and Client secret you found in Step 1.
3. Enter the Logs Ingestion URI you found in Step 2 or DCR logIngestion you found in Step 3.
4. Enter the DCR values immutableId and Log stream name you found in Step 3.
5. Click Save & Test. Fivetran will take it from here and sync your logs to Azure Monitor.

Setup test

Fivetran performs the Connect to Azure Monitor Logs setup test to check service accessibility and the credentials you provided in the setup form.

How to migrate from the legacy Data Collector API table to a DCR-based table

Follow the Microsoft migration guide to migrate your existing custom logs to the DCR-based table.

If you've migrated your existing custom log table, and the table contains the createdAt_t column, you must map a new column, CreatedAt, to the existing createdAt_t column:

1. Add createdAt_t column in stream declaration:

   az monitor data-collection rule update \
     --resource-group "loganalyticsdefaultresources" \
     --name "ft_log_ingest_test_dcr" \
     --add stream_declarations.Custom-ft_log_ingest_test_CL.columns '{"name":"createdAt_t","type":"datetime"}'
   

2. Add column mapping (append your existing logic in transformKql, if necessary):

   az monitor data-collection rule update \
     --resource-group "<Resource group name>" \
     --name "<DCR Name>" \
     --set "data_flows[<Data flow number in array>].transformKql=source | extend createdAt_t = CreatedAt | project-away CreatedAt"