Daily Dev Tips
Hello from the Product Development Team at Qualiteg. TensorRT-LLM is an inference engine from NVIDIA that can be regarded as the successor to FasterTransformer, and it is available as one of the inference engines for our ChatStream. Like vLLM, it adds support for new models quickly and already supports a
![[AI-Driven New Business Creation] The Five Steps of Idea Generation, as Qualiteg Sees Them](https://images.unsplash.com/photo-1496053512767-4fdda61b3c49?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3wxMTc3M3wwfDF8c2VhcmNofDE0fHxzdGVwfGVufDB8fHx8MTcxMzcwMDkxNnww&ixlib=rb-4.0.3&q=80&w=600)
AI-Business
For clients who struggle with idea generation, we explain five steps: set a clear objective, then move through idea creation, deepening, service design, business model design, and evaluation. The process verifies that ideas can stand as a real business, aligned with the agreed objective.
![[Updated May 14, 2024] LLM Inference API Pricing and Inference Speed](/content/images/size/w600/2024/05/LLM--------1.png)
IT & AI Technology
Here is a summary of the pay-as-you-go pricing and generation speeds for using LLMs via API. We plan to update it over time. [API Pricing] shows the output-side price per 1 million tokens. [Generation Speed] is expressed in "tokens/s" (tokens per second), i.e., how many tokens
AI Mathematics
Good morning from the Qualiteg Research Team. This time, let's look at cross-entropy for binary classification. Chapter 7: Cross-Entropy for Binary Classification 7-1. Cross-Entropy for Binary Classification (Single-Sample Version) From here, we would like to derive the cross-entropy for binary classification. Binary classification predicts which of two classes
![[ChatStream] Pin the Revision to Guard Against Unexpected Changes in Input/Output Prompts](/content/images/size/w600/2024/05/-----TIPs--1--1.png)
Daily Dev Tips
Hello, this is the Product Development Department at Qualiteg Inc. During Japan's Golden Week holidays, the tokenizer.json of microsoft/Phi-3-mini-128k-instruct was changed. As a result, prompt parsing failed and chat streaming stopped working. Specifically, the following change was made: https://huggingface.co/microsoft/Phi-3-mini-128k-instruct/commit/8a362e755d2faf8cec2bf98850ce2216023d178a Originally,
DIY GPU Machines
In this installment, we install Anaconda. Python applications are generally run inside a virtual environment created for each application. Two tools are commonly used to build such virtual environments: Anaconda, or Python's built-in venv. This time, we will go with Anaconda. 4.1 Installing Anaconda (the Python environment)
Daily Dev Tips
This post explains how to deal with the following error when you try to pip install bitsandbytes. Could not load bitsandbytes native library: libcusparse.so.11: cannot open shared object file: No such file or directory Traceback (most recent call last): File "/home/mlu/.virtualenvs/ChatStream/lib/python3.10/
![[AI-Driven New Business Creation] Qualiteg's Original Method for Effectively Comparing Customer Problems](https://images.unsplash.com/14/unsplash_5243e9ef164a5_1.JPG?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3wxMTc3M3wwfDF8c2VhcmNofDIxfHxjb21wYXJlfGVufDB8fHx8MTcxMzcwODAxOHww&ixlib=rb-4.0.3&q=80&w=600)
AI-Business
To compare customer problems, research market size, look for synergies with existing businesses, and examine feasibility. Evaluate each problem by market potential, fit with current businesses, technology, and cost, then score and compare the results to clarify which market to enter.
ChatStream Guide
Hello, this is the Product Development Department at Qualiteg Inc. Today we would like to introduce PagedAttention, a technology that is indispensable for commercial LLM serving. Introduction PagedAttention is an extremely important technology for our company. By taking advantage of PagedAttention, you can improve GPU memory utilization during text generation
News
On April 25, 2024, Qualiteg Inc. was accepted into the NVIDIA Inception program. This marks a new step forward for us as we join innovative startups from around the world in developing and bringing advanced technologies to market. What This Means for Our LLM Service Development Being accepted into the
Daily Dev Tips
This post explains how to deal with the error "triu_tril_cuda_template" not implemented for 'BFloat16', which can occur when you specify torch_dtype=torch.bfloat16 while loading a model. Below is the log from when this occurred with Llama 3. File "/home/mlu/
![[ChatStream] A ChatPrompt Class for meta-llama/Meta-Llama-3-8B-Instruct](/content/images/size/w600/2024/04/ChatPrompt--4-.png)
ChatStream Guide
In this post we introduce a ChatPrompt class* for Llama 3, which was released yesterday (April 19, 2024). from chatstream import AbstractChatPrompt SYSTEM_PROMPT = """\ You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any
DIY GPU Machines
Hello! In this installment, we install the NVIDIA display driver so that we can use the GPU on Ubuntu. 3.2 Installing the NVIDIA Display Driver We will install the display driver that supports the GPU we purchased, an NVIDIA RTX 3090 Ti 24GB. 3.2.1 Downloading the Driver
AI Mathematics
Good morning from the Qualiteg Research Team. In this installment, we extend the previous discussion and derive the cross-entropy for multi-class classification with N samples, working through actual data as we go. Chapter 6: Cross-Entropy for Multi-Class Classification (N Samples) In real training, we perform batch learning: several training samples
![[AI-Driven New Business Creation] Qualiteg's Original Method for Effectively Evaluating Customer Problems](https://images.unsplash.com/photo-1544819679-57b273c027a3?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3wxMTc3M3wwfDF8c2VhcmNofDE0fHxldmFsdWF0aW9ufGVufDB8fHx8MTcxMzcwODA4MXww&ixlib=rb-4.0.3&q=80&w=600)
AI-Business
Many companies choose customer problems on a planner's intuition, which often leads them astray. Selection requires a deep grasp of market and user needs and alignment with the business's purpose. Qualiteg recommends choosing problems that fit the purpose agreed with the business owner up front.
![[AI-Driven New Business Creation] The Qualiteg Way: Defining Customer Problems Based on Hypothesis-Exploration Interviews](https://images.unsplash.com/photo-1512758017271-d7b84c2113f1?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3wxMTc3M3wwfDF8c2VhcmNofDJ8fGJ1c2luZXNzJTIwcHJvYmxlbXxlbnwwfHx8fDE3MTM3MDA3MzV8MA&ixlib=rb-4.0.3&q=80&w=600)
AI-Business
In new business consulting, many people want to jump straight into brainstorming. In practice, a fact-based grasp of customer problems leads to better plans. The process covers reconfirming customer needs, extracting insights, and diverging on problems with the How Might We framework.