Qualiteg - Qualiteg Product Development Team

NumPy/PyTorch

Optimizing PyTorch Models: How TorchScript Works and How to Use It

Hello! Today's topic is optimization, an essential step when taking an AI application developed with PyTorch to production. Specifically, we will look at optimizing various trained models using TorchScript. TorchScript Basics 1 What Is TorchScript? TorchScript is a technology that converts PyTorch models into an optimized intermediate representation

NumPy/PyTorch

Mastering Tensor Operations: Memory Efficiency with In-Place Operations

Hello! Today we are covering in-place operations in PyTorch. When building deep learning models, memory management becomes a major challenge. For many of you, the biggest part of that challenge is probably GPU memory. This is where PyTorch's in-place operations come to the rescue. In this article, we

ChatStream Guide

chatstream.net Query Parameter Specification

chatstream.net is a service operated by Qualiteg Inc. that lets you try out the latest publicly available LLMs from around the world as soon as they are released. You can open a particular LLM for an in-depth chat, open several LLMs and use them together, or have LLMs compare

NumPy/PyTorch

Safe Parallel Inference and Performance Optimization for Deep Learning Models

Hello! Today we address one of the questions we hear most often: "Can a single model instance safely run parallel inference?" Safety of Parallel Inference in eval Mode When a PyTorch model has been set to eval mode using model.eval(), it is generally safe for parallel inference.

NumPy/PyTorch

Tensor Operations Worth Mastering: 5 Approaches to Vertically Merging NumPy Arrays

Hello! Today, let's take a look at vertically merging arrays in NumPy. As you know, NumPy is a powerful library for scientific computing in Python. We will walk through five different approaches to vertically merging multiple NumPy arrays into a single larger array. Specifically, we will look at

NumPy/PyTorch

GPU Memory Optimization in Depth: Efficient AI Image Processing and the Peculiarities of the First and Last Batches

Introduction Hello! This is the Qualiteg Product Development Team. At Qualiteg, we research and develop AI characters and AI humans built on LLM technology. When reproducing virtual humans with "human-like" qualities such as facial expressions and gestures, AI-powered image processing—image generation and image editing—becomes essential. AI

PyTorch

PyTorch Versions and Supported NVIDIA GPU Compute Capability Levels (Updated June 2026)

Companies with legacy PyTorch codebases often run into problems like "our old code doesn't run on the latest PyTorch" or "it doesn't run on the latest GPUs (such as the GeForce RTX 50 series, i.e., Blackwell)." In this article, we organize

NumPy/PyTorch

Tensor Operations Worth Mastering: tensor.unsqueeze(0) vs. array[None]

Today we will walk through unsqueeze(0), and along the way touch on a familiar problem many of us run into. Can you tell "value.unsqueeze(0)" and "value[None]" apart at a glance? The former is primarily a PyTorch operation, while the latter belongs to

NumPy/PyTorch

Tensor Operations Worth Mastering: reshape(N, -1)

NumPy reshape: Taking Full Control of Your Data's Shape NumPy's reshape function is a powerful tool for changing the shape of multidimensional arrays. In this article, we cover reshape in detail, from basic usage to more advanced applications, with concrete examples. 1. reshape Basics reshape changes

Python

How to Reset the Anaconda base Environment to Its Initial State

Hello! If you use Anaconda, have you ever accidentally installed extra packages into the base environment? I did exactly that recently—I installed FastAPI into the base environment, and my dependencies ended up in a tangled mess. In this article, we walk through how to safely restore Anaconda's

NumPy/PyTorch

Tensor Operations Worth Mastering: permute(1,0)

This series is for machine learning engineers just getting started: it helps you build a clear mental picture of the tensor operations you use all the time in PyTorch and NumPy. Rather than a reference that aims for strict formal rigor, we will learn through the kinds of code you

LLM Security

LLM-Audit: The Front Lines of LLM Attacks and Defenses

Introduction With the rapid evolution of artificial intelligence, large language models (LLMs) have become indispensable tools for many companies and organizations. Their range of applications—natural language processing, code generation, data analysis, and more—expands daily, driving innovation across a wide range of fields, from streamlining business processes to supporting

Daily Dev Tips

Fixing the ONNX Runtime CUDA Error "libcublasLt.so.11: cannot open shared object file"

Hello! When working with ONNX Runtime, you may run into an error like the following [E:onnxruntime:Default, provider_bridge_ort.cc:1744 TryGetProviderInfo_CUDA] Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory [W:onnxruntime:

LLM

Introducing Llama 3.1, the Latest Large Language Model Series Announced by Meta

On July 23, 2024, Meta announced its latest large language models, the Llama 3.1 series. In this article, we introduce the features and performance of the Llama 3.1 series, along with a hands-on look at using it. We have also covered this topic in the video below, so

LLM

A Deep Dive into Mistral NeMo 12B, the Latest LLM from Mistral AI

Hello. In this post we introduce "Mistral NeMo 12B," the latest LLM from Mistral AI, released on July 19, 2024. We will walk through the model's features and performance, and then try it out in an actual chat session using ChatStream. What Is Mistral NeMo 12B?

LLM

Trying Out Codestral Mamba 7B, an Innovative Code-Generation LLM

Today we tried out "mistralai/mamba-codestral-7B-v0.1" (commonly known as Codestral Mamba 7B), a new code-generation LLM released on July 16, 2024. This model adopts the new Mamba architecture and is published under the Apache 2 license. Performance Approaching SOTA Code-Generation Models Although Codestral 7B is built on