Qualiteg - NumPy/PyTorch - Qualiteg Journal

Daily Dev Tips

How Making Heavy PyTorch CUDA Processing Async Caused a Memory Leak—and How We Fixed It

Hello! This is the Qualiteg Product Development team! This is the story of how simply converting a synchronous method into an asynchronous one (async) hit us with an unexpected memory leak*. It happened while we were developing a video processing system that uses deep learning models. We had a requirement

NumPy/PyTorch

When You Hit a Segmentation Fault in a GPU Service: A Practical Approach from Analysis to Resolution

Hello! Today, let's take a look at analyzing and dealing with segmentation faults in services that combine virtualized environments with GPUs. What a Segmentation Fault Really Is A segmentation fault is an exception raised by the OS when a program attempts to access a protected region of memory.

NumPy/PyTorch

PyTorch 2.6 Security: Safe Practices for Loading Model Checkpoints

Hello! Today's topic: just because an attractive PyTorch checkpoint is being handed around does not mean you should innocently use it — doing so can be seriously dangerous. Have you ever thought about the security risks of loading a model checkpoint? A checkpoint you assumed was just a model

NumPy/PyTorch

The "Unknown Error" Problem on Production PyTorch + CUDA Servers and How to Address It

Hello! This is the Qualiteg Product Development team. Today's topic is a headache familiar to anyone running commercial GPU services: the "CUDA error: unknown error" that suddenly appears after long periods of operation, even though every test passes. Running into it is disheartening, but in the

NumPy/PyTorch

Optimizing PyTorch Models: How TorchScript Works and How to Use It

Hello! Today's topic is optimization, an essential step when taking an AI application developed with PyTorch to production. Specifically, we will look at optimizing various trained models using TorchScript. TorchScript Basics 1 What Is TorchScript? TorchScript is a technology that converts PyTorch models into an optimized intermediate representation

NumPy/PyTorch

Mastering Tensor Operations: Memory Efficiency with In-Place Operations

Hello! Today we are covering in-place operations in PyTorch. When building deep learning models, memory management becomes a major challenge. For many of you, the biggest part of that challenge is probably GPU memory. This is where PyTorch's in-place operations come to the rescue. In this article, we

NumPy/PyTorch

Safe Parallel Inference and Performance Optimization for Deep Learning Models

Hello! Today we address one of the questions we hear most often: "Can a single model instance safely run parallel inference?" Safety of Parallel Inference in eval Mode When a PyTorch model has been set to eval mode using model.eval(), it is generally safe for parallel inference.

NumPy/PyTorch

Tensor Operations Worth Mastering: 5 Approaches to Vertically Merging NumPy Arrays

Hello! Today, let's take a look at vertically merging arrays in NumPy. As you know, NumPy is a powerful library for scientific computing in Python. We will walk through five different approaches to vertically merging multiple NumPy arrays into a single larger array. Specifically, we will look at

NumPy/PyTorch

GPU Memory Optimization in Depth: Efficient AI Image Processing and the Peculiarities of the First and Last Batches

Introduction Hello! This is the Qualiteg Product Development Team. At Qualiteg, we research and develop AI characters and AI humans built on LLM technology. When reproducing virtual humans with "human-like" qualities such as facial expressions and gestures, AI-powered image processing—image generation and image editing—becomes essential. AI

NumPy/PyTorch

Tensor Operations Worth Mastering: tensor.unsqueeze(0) vs. array[None]

Today we will walk through unsqueeze(0), and along the way touch on a familiar problem many of us run into. Can you tell "value.unsqueeze(0)" and "value[None]" apart at a glance? The former is primarily a PyTorch operation, while the latter belongs to

NumPy/PyTorch

Tensor Operations Worth Mastering: reshape(N, -1)

NumPy reshape: Taking Full Control of Your Data's Shape NumPy's reshape function is a powerful tool for changing the shape of multidimensional arrays. In this article, we cover reshape in detail, from basic usage to more advanced applications, with concrete examples. 1. reshape Basics reshape changes

NumPy/PyTorch

Tensor Operations Worth Mastering: permute(1,0)

This series is for machine learning engineers just getting started: it helps you build a clear mental picture of the tensor operations you use all the time in PyTorch and NumPy. Rather than a reference that aims for strict formal rigor, we will learn through the kinds of code you

NumPy/PyTorch

At Inference Time, Use sourceTensor.clone().detach() Instead of torch.tensor(sourceTensor)

Optimizing PyTorch Tensor Operations: Understanding and Resolving a Warning Message Hello! This is the Qualiteg Product Development Team. While working with PyTorch 1.13, we ran into the following warning message: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach(