IT & AI Technology
Hello, this is the Qualiteg Research Team. What Is Speculative Decoding? Speculative decoding is a technique for accelerating inference in large language models (LLMs). It has been reported to speed up most models by roughly 1.4x to 2.0x. In this approach, a small model (the draft model) produces
AI Mathematics
Good morning! This is the Qualiteg Research Team. In this article, we explain how to express the cross-entropy computation using vectors and matrices. Chapter 8: Cross-Entropy and Vector Operations First of all, why would we want to express cross-entropy with vectors and matrices? The reason is that when you actually
ChatStream Guide
[Appendix] Comparison of Penalties Penalty type Purpose How it is applied Example penalty Repetition Penalty Prevents specific tokens or phrases from being repeated. Applies a penalty to the log probabilities (logits) of all previously generated tokens. For example, if a token has already been generated, its log probability is divided
AI Mathematics
Good morning from the Qualiteg Research Team. This time, let's look at cross-entropy for binary classification. Chapter 7: Cross-Entropy for Binary Classification 7-1. Cross-Entropy for Binary Classification (Single-Sample Version) From here, we would like to derive the cross-entropy for binary classification. Binary classification predicts which of two classes
ChatStream Guide
Hello, this is the Product Development Department at Qualiteg Inc. Today we would like to introduce PagedAttention, a technology that is indispensable for commercial LLM serving. Introduction PagedAttention is an extremely important technology for our company. By taking advantage of PagedAttention, you can improve GPU memory utilization during text generation
AI Mathematics
Good morning from the Qualiteg Research Team. In this installment, we extend the previous discussion and derive the cross-entropy for multi-class classification with N samples, working through actual data as we go. Chapter 6: Cross-Entropy for Multi-Class Classification (N Samples) In real training, we perform batch learning: several training samples
AI Mathematics
Good morning! This is the Research Department at Qualiteg Inc. In this installment, we will calculate the cross-entropy used for multi-class classification. Chapter 5: Calculating Cross-Entropy for Multi-Class Classification (Single-Sample Version) First, let us restate the cross-entropy function (single-sample version). $$ \ - \log L=\sum_{k=1}^{K} t_{k} \log
AI Mathematics
Good morning! This is the Qualiteg Research Department. In this installment, we will derive cross-entropy from the likelihood function. Chapter 4: Deriving Cross-Entropy from the Likelihood Function So far, we rolled a die 20 times, counted the events that occurred (whether the result was 1, 2, ..., or 6), and summarized
AI Mathematics
Good morning. This is the Qualiteg Research Department. Let's pick up right where we left off last time. Chapter 2: Interpreting Classification as Predicting Probabilities In the classification problems we want to solve with deep learning, logistic regression, and so on, we predict which class an input belongs
AI Mathematics
Good morning from the Qualiteg Research Department. Starting today, we would like to take a thorough look at cross-entropy. Are there two formulas for the cross-entropy function? In this series, we take up the cross-entropy function, which is commonly used as the loss function for classification problems in machine learning.
AI Mathematics
Good morning. This is the Research Team at Qualiteg. Today we present the second half of our discussion on differentiating logarithmic functions. By the end of this installment, you should have a solid grasp of how to differentiate logarithmic functions. 2. Logarithm Identities First, let's review the basic
AI Mathematics
Good morning from the Qualiteg Research Team. Starting today, we will spend two articles learning about logarithmic functions. To be precise, we will learn how to differentiate logarithmic functions. What was a logarithmic function again? First of all, a logarithm is a way of finding "how many times a
AI Mathematics
Good morning from the Research Department at Qualiteg Inc. Today, let's study exponential functions. They come in handy when deriving the differentiation formula for logarithmic functions, which is why it's their turn now. What is an exponential function? An exponential function is a mathematical expression that
AI Mathematics
Hello everyone, this is the Research Department at Qualiteg. When reading papers on LLMs, do you ever feel dizzy from the sheer number of mysterious formulas and symbols? I studied neural networks back in my student days, and I remember being taken aback by how the mathematical background needed to