Hello!

Today, let's take a look at analyzing and dealing with segmentation faults in services that combine virtualized environments with GPUs.

What a Segmentation Fault Really Is

A segmentation fault is an exception raised by the OS when a program attempts to access a protected region of memory.
In our case, it occurred when several GPU services (that is, processes using the GPU) were running and we restarted one of them.
It does not happen every time. In fact, it occurs perhaps once in several hundred startups — but even a single occurrence can be devastating. The failure of one GPU service becomes a SPOF (single point of failure) that affects the entire service. Worse, once a segmentation fault occurred, the offending process would never start again — a truly troublesome phenomenon.

This characteristic — "it normally works fine, then suddenly stops working" — is what makes debugging extremely difficult.

In our experience, the combination of GPU and virtualization noticeably raises the probability of occurrence compared to traditional C++ applications.

To summarize the characteristics:

• High frequency in GPU-based services (image recognition, machine learning workloads, and so on)
• Tends to occur suddenly after long periods of operation
• The difference from a CUDA unknown error: the latter may work if you wait and retry, but with a segmentation fault, the process never starts again.

(For readers with deep expertise who are itching to object: "segmentation fault" is admittedly a very coarse-grained error category, so please read this as "the one particular case we encountered.")

Common causes of segmentation faults include:

• Dereferencing a NULL pointer
• Accessing freed memory
• Stack overflow
• Writing to read-only memory

In a GPU-plus-virtualization environment, however, most of the time you simply cannot tell which of these is to blame.

A Real-World Case

Now, let us walk through a case we actually experienced recently.

When we tried to run a GPU-based application, the following log was output.

import 'scipy.special._orthogonal' # <_frozen_importlib_external.SourceFileLoader object at 0x7fc006e862b0>
# /home/user/anaconda3/envs/image_recognition_env/lib/python3.9/site-packages/scipy/special/__pycache__/_spfun_stats.cpython-39.pyc matches ...
...
import 'yaml' # <_frozen_importlib_external.SourceFileLoader object at 0x7fc003d215e0>
...
Segmentation fault (core dumped)

Core Dump Analysis: Getting to the Heart of the Problem

In GPU application development and operations — especially with C/C++ programs and Python C extension modules — the unexpected "Segmentation fault" error can strike. Even in Python applications that seemingly have nothing to do with C, the underlying libraries or the CUDA stack can be the cause (these days, that is the most common pattern).

This error usually appears with the message "Segmentation fault (core dumped)."

So what exactly is this "core dump"?

What is a core dump?

A core dump is a file capturing the memory state of a program at the moment it crashed. Analyzing it lets you see in detail where and in what state the program stopped, which helps you pinpoint the root cause.

STEP 1: Enable Core Dumps

By default, core dump generation may be restricted. First, lift that restriction.

# Remove the core dump size limit
ulimit -c unlimited

# Specify where core dumps are saved and how they are named (Ubuntu/Debian)
echo '/tmp/core.%e.%p.%t' | sudo tee /proc/sys/kernel/core_pattern

With this in place, on a crash a core dump file is saved in the /tmp directory, with a name containing the program name (%e), process ID (%p), and timestamp (%t).

STEP 2: Analyze the Core Dump with GDB

Use the GNU Debugger (GDB) to analyze the generated core dump.

# Analyze the core dump
# Example: for a Python program
gdb python /tmp/core.python.12345.1620000000

When GDB starts, you get a prompt. From there, use the following commands to dig into the problem.

STEP 2-1. Show the backtrace (stack trace)

(gdb) bt full

This command displays a detailed stack trace showing where the program stopped.

STEP 2-2. Move to a specific frame

When you want to drill into a particular function or operation in the stack trace, move to it by specifying the frame number.

(gdb) frame 3

STEP 2-3. Inspect variables

You can inspect the values of variables within a specific frame.

(gdb) print variable_name

STEP 2-4. Inspect memory contents

You can directly examine what is stored at a memory address.

(gdb) x/10x memory_address

An Example of a Real Error Case

In a common real-world scenario involving Python C extensions or GPU libraries, you might get a backtrace like the following.

#0  0x00007f92d3b7c7a0 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007f92d3b7e8fa in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007f92d5a1c063 in THCudaCheck () from /usr/local/lib/python3.9/site-packages/torch/lib/libtorch_cuda.so
#3  0x00007f92d5a2e4f5 in cudaLaunchKernel () from /usr/local/lib/python3.9/site-packages/torch/lib/libtorch_cuda.so

We can see that the problem occurred when a CUDA kernel was launched.

In cases like this involving external libraries, the cause is often a library configuration issue or a GPU driver version mismatch — but fully identifying the root cause is, practically speaking, nearly impossible (time-wise).

STEP 3: Analyze System Logs

Beyond core dump analysis, system-level logs also provide useful information. Check the relevant logs with the following commands.

dmesg | grep -i segfault
journalctl | grep -i segfault

This retrieves kernel-level messages, which can sometimes reveal information about memory issues or problems specific to virtualized environments. In practice, though, you rarely have the luxury of going this far.

Problems Specific to Virtualized Environments and Countermeasures

These days, systems commonly run in multi-layered, millefeuille-like virtualization stacks — Docker, VMs, and so on.
In such environments, privilege levels, resource limits, and memory access issues become far more complicated — reliably draining your motivation to debug.

STEP 1. Deal with the ptrace problem in Docker

When debugging inside a Docker container, ptraceattempting to use a debugger that relies on system calls (GDB, for example) can fail with a permission error.
This is because Docker, for security reasons, disables the SYS_PTRACE capability by default.

To solve this, you need to explicitly allow SYS_PTRACE. You can configure it with commands like the following.

# Enable SYS_PTRACE when starting the container
docker run --cap-add=SYS_PTRACE image_name

# Add SYS_PTRACE to an already-running container
docker update --cap-add=SYS_PTRACE container_name

With this, the debugger can run safely and correctly inside the container.

STEP 2. Capture stack traces with pstack

Checking what state a process is in when a problem occurs is a fundamental part of debugging.pstack is a tool that makes it easy to capture the stack trace of a running process. It is especially useful for investigating frozen processes or the causes of abnormal termination.

You can install and use pstack as follows.

sudo apt install pstack
pstack process_id

Running the command above immediately displays the current stack trace of the specified process. From the output, you can review the function call history and narrow down where the bug occurred.

STEP 3. Lightweight diagnosis of running processes and memory snapshots

When a running process starts behaving unexpectedly, taking a complete snapshot including its memory state allows for detailed investigation later. The tool for this is gcore.gcore saves the current state of the specified process as a core dump.

Usage is as follows.

gcore process_id  # Generate a core dump of the specified process

Once the core dump is generated, you can later use a debugger such as GDB to deeply analyze the process's memory contents and stack state at that moment. This lets you work on pinpointing the problem without stopping the running process.

As you can see, debugging problems specific to virtualized environments can be handled fairly effectively with the right tools and a healthy dose of grit. In Docker in particular, capability management is key, and combining it with process diagnostic tools (pstack and gcore) can — with persistence — let you identify and resolve the problem.

System Design for Preventing Recurrence

For sudden, hard-to-predict errors like segmentation faults, completely preventing them is difficult. The more realistic approach is to build mechanisms that minimize the impact when they do occur.

Implementing restarts and a watchdog process

You may groan, "That's your answer?!" — but in the end, a restart is the most effective remedy.

For GPU-based services, the most basic and effective way to prepare for the risk of a process dying suddenly is to implement a watchdog that monitors process liveness and automatically attempts a restart when the process stops.

Below is a simple example of a watchdog implemented in Python.

import subprocess
import time
import logging

def run_with_watchdog(command, max_retries=3, wait_time=5):
    for attempt in range(max_retries):
        try:
            process = subprocess.Popen(command, shell=True)
            return_code = process.wait()
            if return_code == 0:
                return True
            logging.warning(f"Process exited with return code {return_code}. Retry {attempt + 1}/{max_retries}")
        except Exception as e:
            logging.error(f"An exception occurred: {e}. Retry {attempt + 1}/{max_retries}")
        time.sleep(wait_time)
    return False

# Usage example
run_with_watchdog("CUDA_VISIBLE_DEVICES=1 python image_recognition_server.py")

A watchdog like this is especially useful in production, allowing recovery from temporary outages caused by segmentation faults.

Service supervision with systemd

On Linux, the recommended, more robust, and standard approach to process management is automatic service restart via systemd.

Here is an example systemd configuration.

# /etc/systemd/system/image-recognition.service
[Unit]
Description=Image Recognition Service
After=network.target

[Service]
User=app_user
WorkingDirectory=/opt/app
ExecStart=/usr/bin/python3 /opt/app/image_recognition_server.py
Restart=on-failure
RestartSec=5s
Environment="CUDA_VISIBLE_DEVICES=0"
Environment="PYTHONPATH=/opt/app"

[Install]
WantedBy=multi-user.target

With systemd, the process is automatically restarted when it terminates abnormally, keeping the service available.

Monitoring GPU and process memory usage

By monitoring GPU and process memory usage, you can detect memory fragmentation and resource exhaustion early.

# Check GPU memory status
nvidia-smi

# Check the process memory map
cat /proc/<PID>/maps

Using memory error detection tools

# AddressSanitizer (detects memory leaks and overflows)
PYTHONMALLOC=malloc ASAN_OPTIONS=detect_leaks=0 python -m pip install --no-binary :all: problematic_package

# Detailed memory leak investigation with Valgrind
valgrind --leak-check=full python script.py

# Capture a stack trace of a running process
pstack process_id

A Practical Solution

After all this discussion, the bottom line: a restart is still the strongest remedy.

But even a restart should follow a staged approach.

STEP 1. Re-run with minor environment variable adjustments

CUDA_VISIBLE_DEVICES=1 python script.py
CUDA_VISIBLE_DEVICES=0 PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:128 python script.py

STEP 2. Restart the virtual environment or container

For Docker:

docker restart container_name

For WSL:

wsl --shutdown

STEP 3. Restart the entire host system

Linux:

sudo reboot

Windows:

Choose Restart

Try restarts in stages like this.
For GPU + virtualized environment + segmentation fault situations, in our experience the best-balanced option is STEP 2: restarting the container or virtual environment. As in our case, when multiple processes share a single GPU's memory and one of them hits a segmentation fault, restarting just that process often fails to recover it — presumably because complex memory fragmentation across multiple processes has built up inside the GPU. Restarting the container or virtual environment, by contrast, forcibly detaches every process holding GPU memory at once. It also restores service considerably faster than a full OS reboot, which is another attractive point.

Preventing Recurrence with Regular Maintenance

Of course, the most fundamental way to head off sudden problems is to schedule regular restarts and maintenance in the first place.

Conclusion

In this article, we started from the segmentation fault and
took a quick tour of its causes, analysis methods, remedies when it strikes, and prevention. With segmentation faults, fully identifying the root cause is not always possible, but by combining layered approaches on both the analysis and prevention fronts, we will keep working to maintain high practicality and system availability. Yes, this article too ended with "so it's a restart after all" — but we believe the essence of engineering is choosing simple, robust solutions to complex problems, and we will keep striving to minimize the impact of such issues through the right combination of monitoring, automation, and regular maintenance.

Related Articles

The "Unknown Error" Problem on Production PyTorch + CUDA Servers and How to Address It
https://journal.qualiteg.com/how_to_address_cuda_unknown_error/