LLM

[vLLM] How to Fix "To use CUDA with multiprocessing, you must use the 'spawn' start method"

Qualiteg Product Development Team

Feb 10, 2025 — 1 min read

When using vLLM on WSL and trying to serve a single LLM across multiple GPUs with tensor parallelism, you will often run into an error like the following.

RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method

A typical scenario where you encounter this is when you specify the tensor parallelism option in vLLM's startup options, as shown below.

--tensor-parallel-size 2

In other words, the error is telling you to use "spawn" instead of "fork" when using CUDA with multiprocessing.

To tell vLLM this, set the environment variable as shown on the second line below, and vLLM will use "spawn".

export CUDA_VISIBLE_DEVICES=0,1
export VLLM_WORKER_MULTIPROC_METHOD=spawn

Building a Coding Agent from Scratch, Part 3: Count Idle Turns, and Cap the Thinking Without Cutting It

Building a Coding Agent from Scratch, Part 3: "It Didn't Stop" Is Not "It Made Progress." Count Idle Turns, and Cap the Thinking Without Cutting It

300 turns, zero APIs. Re-reading the log showed 234 turns had never called a tool. We cover the idle rate as a metric, idling that shifts form each time one is closed, how turning thinking off dropped 95 to 71 without saving time, and counting the other side after widening the output window.

Ubuntu's Automatic Updates Bumped the Kernel Alone, and the GPU Vanished After a Reboot: Why the NVIDIA Module Gets Left Behind, and Three Ways to Prevent It

Reboot an Ubuntu 24.04 GPU server and nvidia-smi stops working. The cause: unattended-upgrades updated the kernel alone and skipped the NVIDIA module over a dependency it could not resolve. How we traced it in the logs, recovered with one install, and compared three fixes including apt-mark hold.

Building a Coding Agent from Scratch, Part 2: The Completion Gate and What to Write in a Pushback

Building a Coding Agent from Scratch, Part 2: "Done" Means "It Runs," Not "It's Written." The Completion Gate and What to Write in a Pushback

The model's "Done" slipped through in five ways, from a bare syntax check to tests with no assert. We describe, with numbers, a completion gate that closes in three stages, and why a pushback should contain the failing lines and how to investigate, but never the answer.

Jev by TypeSafe AI: What It Is and How Well It Works, Measured Over 301 API Calls

Jev returns typed decisions instead of text. Signups reopened on September 28 with no free credit for direct accounts. We put $10 in, called it 301 times from Python, and measured accuracy on Japanese tickets, guardrails and command approval, plus response time and cost (about one yen).

Read more

Building a Coding Agent from Scratch, Part 3: "It Didn't Stop" Is Not "It Made Progress." Count Idle Turns, and Cap the Thinking Without Cutting It

Ubuntu's Automatic Updates Bumped the Kernel Alone, and the GPU Vanished After a Reboot: Why the NVIDIA Module Gets Left Behind, and Three Ways to Prevent It

Building a Coding Agent from Scratch, Part 2: "Done" Means "It Runs," Not "It's Written." The Completion Gate and What to Write in a Pushback

Jev by TypeSafe AI: What It Is and How Well It Works, Measured Over 301 API Calls