Column $4097 Does Not Exist

Issue

An upload fails with the following error:

{
    "status" : "FAILURE_WITH_TASK",
    "reason" : "net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error:\ncolumn '$4097' does not exist"
}

Environment

Destination: Snowflake

Resolution

The upload contains more than 4096 columns. To resolve the error, reduce the number of columns.

Cause

Snowflake supports a maximum of 4096 columns when data is uploaded using the COPY INTO <TABLE> command from external or internal storage. If the number of columns exceeds this limit, the upload fails and returns the error message shown above.