LLM
Hello! Welcome to Part 2 of the LLM Inference Infrastructure Provisioning Course. LLM Inference Infrastructure Provisioning Course: Series Index * Part 1: Basic Concepts and Inference Speed * Part 2: Estimating Request Volume for an LLM Service * Part 3: Estimating Inference-Time Memory Consumption for Your Model * Part 4: Selecting an Inference Engine
AI Mathematics
Hello! Today we walk through the algorithms and know-how behind stabilizing auto-framed video. Chapter 1: Background and Objectives When shooting bust-up video — especially talk footage for online meetings or YouTube — natural movements such as nodding or shaking the head often cause the neck area and shoulders to shift up and
LLM
Hello! Today we begin a multi-part series on inference infrastructure provisioning — GPU provisioning in particular — for building your own LLM service in-house. Introduction As LLMs have evolved, the use of public LLMs such as ChatGPT and Claude has spread rapidly through the enterprise. Forward-looking companies, however, are moving beyond these
NumPy/PyTorch
Hello! Today, let's take a look at analyzing and dealing with segmentation faults in services that combine virtualized environments with GPUs. What a Segmentation Fault Really Is A segmentation fault is an exception raised by the OS when a program attempts to access a protected region of memory.
Daily Dev Tips
Hello! Today's topic is the conda command, a tool many of us rely on every day. Here are some tips for shell scripts and batch tasks that enter a conda virtual environment, run some processing, and then return. In AI development, Anaconda and its core conda package manager
Daily Dev Tips
Hello! Today we are talking about Node.js, which we often use as the front-end server for AI systems. With the spread of AI models, we increasingly find ourselves handling very large data files. Model files for LLMs and training datasets can reach several gigabytes—sometimes tens or even hundreds
AI Mathematics
Hello! This is the Qualiteg Research Team! Today we will explore information theory—a concept that is also extremely important in AI—using "Hit & Blow," a game included in the popular Nintendo Switch title "Clubhouse Games: 51 Worldwide Classics," as our subject! Introduction "Hit
IT & AI Technology
Maximum Supported Users Calculator Calculates the maximum number of users supported by a GPU conversion server that can process only one job at a time Processing time (t_p) min/job Time required for one conversion job Target system utilization (ρ) 0 to 1 Target utilization for stable operation (recommended:
Daily Dev Tips
Hello! Windows Subsystem for Linux (WSL2) is a wonderful feature that lets you run a Linux environment on Windows, but some users run into DNS resolution problems. This article covers the symptoms and an effective way to fix them. Test Environment The method described in this article was verified with
NumPy/PyTorch
Hello! Today's topic: just because an attractive PyTorch checkpoint is being handed around does not mean you should innocently use it — doing so can be seriously dangerous. Have you ever thought about the security risks of loading a model checkpoint? A checkpoint you assumed was just a model
Daily Dev Tips
This is an issue I have run into a few times after repeatedly installing and removing various versions of Linux on WSL. Encountered errors while loading user settings Found a profile with an invalid "icon". Defaulting that profile to have no icon. Make sure when setting an "
NumPy/PyTorch
Hello! This is the Qualiteg Product Development team. Today's topic is a headache familiar to anyone running commercial GPU services: the "CUDA error: unknown error" that suddenly appears after long periods of operation, even though every test passes. Running into it is disheartening, but in the
Daily Dev Tips
Hello! When backing up or archiving a folder that contains multiple large files, we often pack them into a ZIP file. However, recompressing files that are already compressed (videos, images, PDFs, and so on) can take a long time while yielding little benefit—or make the compression step painfully slow.
![[Explainer] What Is the Tekken Tokenizer? Inside the New-Generation Tokenizer Adopted by Mistral](/content/images/size/w600/2025/02/DALL-E-2025-02-12-10.17.55---A-photorealistic-image-of-a-gladiator-holding-a-sword.-The-gladiator-is-wearing-traditional-Roman-armor-with-a-helmet--chest-plate--and-a-short-tunic.webp)
LLM
Hello! Today we are going to talk about Tekken! When you hear "Tekken," what comes to mind? The fighting game Tekken (鉄拳, "iron fist"), perhaps? Personally, I thought of the Long Swordsman (鉄剣戦士, "iron-sword warrior" in the Japanese edition) from Age of Empires, a
LLM
Hello! On January 30, 2025, Mistral AI announced its new language model, Mistral Small v3. Despite a relatively small parameter count of 24B, the model achieves performance comparable to large-scale models of 70B or more. It also advertises Japanese-language support, making it a highly anticipated compact model. https://huggingface.co/
![[vLLM] How to Fix "To use CUDA with multiprocessing, you must use the 'spawn' start method"](https://images.unsplash.com/photo-1597872200969-2b65d56bd16b?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3wxMTc3M3wwfDF8c2VhcmNofDh8fGdwdXxlbnwwfHx8fDE3MzkyMzYxNTh8MA&ixlib=rb-4.0.3&q=80&w=600)
LLM
When using vLLM on WSL and trying to serve a single LLM across multiple GPUs with tensor parallelism, you will often run into an error like the following. RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method A