Sync Failed Due to Segmentation Fault
Issue
A sync fails abruptly and the sync logs contain one or more of the following:
Fatal Python error: Segmentation fault
Segmentation fault (core dumped)
The logs may also include a thread stack trace showing the file and line number in your connector code where the crash occurred. In some cases, the sync logs may contain no output at all, or only the shell-level message such as /app/init.sh: line N: <pid> Segmentation fault. This happens when the crash occurs so early or so abruptly that no stack trace can be captured before the process is terminated. In these cases, use fivetran debug locally to attempt reproduction, as the local terminal will capture any available output directly.
Environment
Fivetran Connector SDK
Resolution
Use the stack trace in the connection's sync logs to identify the crashing location and apply one of the following fixes:
Create resources per thread — if your connector uses multithreading, such as
ThreadPoolExecutor, ensure that database connections, cursors, and any other C-backed objects are created inside each thread rather than shared across threads.Verify resource initialization — check that connections are fully established before use. Calling methods on a partially initialized or failed connection — for example, calling
.cursor()before the connection is open — can cause a crash inside the underlying C library.Check library compatibility — ensure that all native (C extension) packages in your
requirements.txtare compatible with each other and with the Python version used for connection. Version mismatches in libraries such as database drivers, compression libraries, or ML libraries with C backends are a common cause.Reproduce locally — run
fivetran debugto reproduce the issue locally. The full thread stack trace will be printed to the terminal, showing the exact file and line number where the crash occurred.Isolate the failing library — if the stack trace points into a third-party library, try replacing or upgrading it. Remove suspect packages from
requirements.txtone at a time to identify the culprit.
Cause
A segmentation fault means the operating system terminates the Python process due to an illegal memory access inside a C-level function. Because the crash happens inside native code, Python cannot raise a normal exception — the operating system immediately ends the process with exit code 139.
Common causes include:
- Non-thread-safe C extensions — libraries such as database drivers and compression buffers often use internal C-level state that is not safe to share across threads. Concurrent access without proper locking causes memory corruption and a crash.
- Uninitialized or NULL native resources — a C extension may return a NULL or uninitialized pointer when a connection is not properly set up. Writing to or reading from that pointer causes an immediate crash.
- Incompatible native library versions — a mismatch between a Python C extension and the shared library it links against can lead to undefined behavior and crashes.
- Memory exhaustion in native code — C extensions allocate memory directly at the OS level, outside Python's memory manager. If a C extension exhausts available memory, the operating system may terminate the process with a segmentation fault rather than a clean out-of-memory error.
- Stack overflow from deep recursion in C — deeply recursive calls into C extensions can exhaust the native call stack. Unlike Python recursion, this does not raise a
RecursionError— it crashes the process immediately. - Use-after-free — a C extension holds a raw pointer to a Python object that has since been garbage collected. When the extension later accesses that freed memory, it causes a crash.
- Forking after importing a C extension — using
multiprocessingwith the defaultforkstart method after a C extension has been imported copies the parent process's internal C-level state into child processes. If that state is not fork-safe — for example, open file descriptors, locks, or SSL contexts — the child process will crash unpredictably. Usespawnas the multiprocessing start method instead. - Signal handler conflicts — some C extensions install their own low-level signal handlers. These can interfere with Python's signal handling, causing the process to crash on signals that would otherwise be handled gracefully.
- Bug in the C extension itself — the third-party library may have a known bug that is triggered by a specific data shape, value, or call sequence. Check the library's issue tracker for known segfault reports and upgrade to a fixed version if available.