AI Mathematics
Hello from the Qualiteg Research Team! Today we explain Direct Preference Optimization (DPO)—proposed in 2023 by the research team of Rafael Rafailov, Archit Sharma, and colleagues—from the fundamentals through to its applications. The method was introduced in the paper "Direct Preference Optimization: Your Language Model is Secretly
IT & AI Technology
Hello! This is the Qualiteg Research Team! Today we would like to talk about image generation technology. As many of you know, AI-based image generation technology has evolved rapidly over the past decade. It is now 2024—exactly ten years since GANs were introduced in 2014. While text-based generative AI
![[LLM Security] How to Detect Hallucinations](https://images.unsplash.com/photo-1488272690691-2636704d6000?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3wxMTc3M3wwfDF8c2VhcmNofDI5fHxoYWNrZXJ8ZW58MHx8fHwxNzIyOTA1NTE5fDA&ixlib=rb-4.0.3&q=80&w=600)
LLM Security
Hello, this is the Qualiteg Research Team. Today, we walk through the following paper on hallucination detection in RAG systems and share our thoughts on LLMs built specifically to detect hallucinations. "Lynx: An Open Source Hallucination Evaluation Model" https://arxiv.org/pdf/2407.08488 Overview The paper describes
IT & AI Technology
Hello, this is the Qualiteg Research Team. What Is Speculative Decoding? Speculative decoding is a technique for accelerating inference in large language models (LLMs). It has been reported to speed up most models by roughly 1.4x to 2.0x. In this approach, a small model (the draft model) produces
AI Mathematics
Good morning! This is the Qualiteg Research Team. In this article, we explain how to express the cross-entropy computation using vectors and matrices. Chapter 8: Cross-Entropy and Vector Operations First of all, why would we want to express cross-entropy with vectors and matrices? The reason is that when you actually
IT & AI Technology
Hello! This is the Qualiteg Tech Blog. [Updated July 2, 2024] Today we will cover the features and pricing of the Google Gemini 1.5 Pro / Gemini 1.5 Flash models, and how to generate text with them using Python. What is Google Gemini? Google Gemini is a generative AI
AI Mathematics
Good morning from the Qualiteg Research Team. This time, let's look at cross-entropy for binary classification. Chapter 7: Cross-Entropy for Binary Classification 7-1. Cross-Entropy for Binary Classification (Single-Sample Version) From here, we would like to derive the cross-entropy for binary classification. Binary classification predicts which of two classes
AI Mathematics
Good morning from the Qualiteg Research Team. In this installment, we extend the previous discussion and derive the cross-entropy for multi-class classification with N samples, working through actual data as we go. Chapter 6: Cross-Entropy for Multi-Class Classification (N Samples) In real training, we perform batch learning: several training samples
AI Mathematics
Good morning! This is the Research Department at Qualiteg Inc. In this installment, we will calculate the cross-entropy used for multi-class classification. Chapter 5: Calculating Cross-Entropy for Multi-Class Classification (Single-Sample Version) First, let us restate the cross-entropy function (single-sample version). $$ \ - \log L=\sum_{k=1}^{K} t_{k} \log
AI Mathematics
Good morning! This is the Qualiteg Research Department. In this installment, we will derive cross-entropy from the likelihood function. Chapter 4: Deriving Cross-Entropy from the Likelihood Function So far, we rolled a die 20 times, counted the events that occurred (whether the result was 1, 2, ..., or 6), and summarized
AI Mathematics
Good morning. This is the Qualiteg Research Department. Let's pick up right where we left off last time. Chapter 2: Interpreting Classification as Predicting Probabilities In the classification problems we want to solve with deep learning, logistic regression, and so on, we predict which class an input belongs
![[LLM Security] Llama Guard: A First Step Toward AI Safety](https://images.unsplash.com/photo-1530633866995-0ad6c385894d?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3wxMTc3M3wwfDF8c2VhcmNofDV8fHNoaWVsZHxlbnwwfHx8fDE3MjI5MjQ0Mjl8MA&ixlib=rb-4.0.3&q=80&w=600)
LLM Security
Hello, this is the Qualiteg Research Department. Today we will explain Llama Guard, which was announced at the end of last year (December 2023). In recent years, AI technology has advanced remarkably and its range of applications continues to grow. At the same time, however, concerns about the safety of
AI Mathematics
Good morning from the Qualiteg Research Department. Starting today, we would like to take a thorough look at cross-entropy. Are there two formulas for the cross-entropy function? In this series, we take up the cross-entropy function, which is commonly used as the loss function for classification problems in machine learning.
AI Mathematics
Good morning. This is the Research Team at Qualiteg. Today we present the second half of our discussion on differentiating logarithmic functions. By the end of this installment, you should have a solid grasp of how to differentiate logarithmic functions. 2. Logarithm Identities First, let's review the basic
![[LLM Security] Zero-Resource Black-Box Hallucination Detection](https://images.unsplash.com/photo-1444096443752-cfef93909ac3?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3wxMTc3M3wwfDF8c2VhcmNofDIxfHwlRTklOUMlQTd8ZW58MHx8fHwxNzIyOTMyMTk2fDA&ixlib=rb-4.0.3&q=80&w=600)
LLM Security
Hello from the Qualiteg Research Team. In this article, we walk through the following paper, which proposes a method for detecting hallucinations in a "zero-resource" setting, i.e., without relying on external information such as a database. SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models https:
AI Mathematics
Good morning from the Qualiteg Research Team. Starting today, we will spend two articles learning about logarithmic functions. To be precise, we will learn how to differentiate logarithmic functions. What was a logarithmic function again? First of all, a logarithm is a way of finding "how many times a