New Column Contains Null Values in the Destination
Issue
After adding a new column to a source table, the column appears in the destination but existing rows show null values.
Environment
- Connector: PostgreSQL
- Incremental update method: Query-Based
Resolution
To resolve this issue, consider the following options:
- In Fivetran, re-sync the table to ensure the new column is populated for existing records and schema changes are applied. To learn how, see How to Trigger Historical Re-Syncs for Fivetran Tables.
- Consider switching to the Logical replication update method with the
pgoutputplugin. Logical replication tracks DDL changes more reliably and doesn't rely on XMIN-based row updates. - Minimally update existing rows to bump their XMIN value. This causes Fivetran to re-sync the updated rows, populating the new column without a full table re-sync.
Cause
The Query-Based update method relies on the xmin system column to detect row changes. When you add a new column, PostgreSQL updates the table structure but doesn't modify existing rows, so their xmin values remain unchanged.
When this happens, we don't detect row-level changes or re-sync those rows. As a result, the new column appears in the destination, but existing rows contain null values until you update them or re-sync the table.