Some PostgreSQL Numeric Columns Are Replicated as STRING Columns in BigQuery
Issue
Some PostgreSQL numeric columns replicated as STRING columns in BigQuery.
Environment
- Connector: PostgreSQL
- Destination: BigQuery
Resolution
To resolve this issue:
- In PostgreSQL, alter the affected numeric column to use explicit precision and scale. For example, change the column type to numeric(18,6).
- In Fivetran, go to your PostgreSQL connection page.
- Select the Schema tab.
- Locate and hover over the affected table.
- On the right side of the screen, click Resync.
- In the confirmation pop-up window, click Start re-sync.
If there is existing data that exceeds the BigQuery NUMERIC limits, clean or transform the data before re-syncing. For more information, see Re-sync tables.
Cause
This issue occurs when the PostgreSQL numeric columns are created without an explicit scale and precision. When this happens, the columns allow arbitrary size and precision, which may exceed the BigQuery NUMERIC limits of 38 decimal digits and 9 decimal places.
To prevent data loss, Fivetran replicates PostgreSQL numeric columns without explicit precision or scale as STRING columns in BigQuery.