ChatStream
Hello! Today we tried out Llama-3-Elyza-JP-8B. Announced just yesterday, on June 26, 2024 (https://prtimes.jp/main/html/rd/p/000000046.000047565.html), Llama-3-Elyza-JP-8B is a Japanese LLM whose 70B version is reported to outperform GPT-4. In this post, we share our experience loading Llama-3-Elyza-JP-8B into our own Playground environment
Daily Dev Tips
How to resolve the following error File "/venv/Lib/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/venv/Lib/site-packages/transformers/generation/utils.py", line 1744, in generate model_kwargs["past_key_values"] = self._get_cache( ^^^^^^^^^^^^^^^^ File
![[AI-Driven New Business Creation] Qualiteg Selection, Idea Creation #1: How to Use Qualiteg-Style Online Brainstorming](https://images.unsplash.com/photo-1588856122867-363b0aa7f598?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3wxMTc3M3wwfDF8c2VhcmNofDF8fGJyYWluc3Rvcm1pbmd8ZW58MHx8fHwxNzEzNzAxNTQ4fDA&ixlib=rb-4.0.3&q=80&w=600)
AI-Business
Hello to everyone visiting the Qualiteg blog. I'm Michele. In my work on AI-driven new businesses and marketing, there are questions I hear from clients again and again. Through this blog, I would like to offer my own answers to the challenges that anyone considering an AI-based business
Daily Dev Tips
This is a list of NVIDIA GPU Compute Capability levels. The various acceleration features supported by inference engines depend on the Compute Capability level, since the hardware accelerators and dedicated functions available differ by level. Data Center / Professional GPUs GeForce GPU Capability Level Architecture - GeForce RTX 5090 120 Blackwell
![[AI-Driven New Business Creation] Why Qualiteg Believes in Using Idea Creation Frameworks](https://images.unsplash.com/photo-1508004680771-708b02aabdc0?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3wxMTc3M3wwfDF8c2VhcmNofDJ8fGZyYW1lfGVufDB8fHx8MTcxMzcwMTQ2NHww&ixlib=rb-4.0.3&q=80&w=600)
AI-Business
Hello to everyone visiting the Qualiteg blog. I'm Michele. In my work on AI-driven new businesses and marketing, there are questions I hear from clients again and again. Through this blog, I would like to offer my own answers to the challenges that anyone considering an AI-based business
IT & AI Technology
Hello, this is the Qualiteg Research Team. What Is Speculative Decoding? Speculative decoding is a technique for accelerating inference in large language models (LLMs). It has been reported to speed up most models by roughly 1.4x to 2.0x. In this approach, a small model (the draft model) produces
Daily Dev Tips
In this article, we introduce sample code for using the OpenAI API. The OpenAI API is not limited to OpenAI's own service; other inference engines such as vLLM also expose OpenAI-compatible API servers, and it is becoming the de facto standard for LLM serving APIs. So it is
DIY GPU Machines
Without further ado, let's run some Python code on the GPU. 4.3 Writing GPU-Accelerated PyTorch Code in Jupyter Notebook STEP 1: Open a terminal and create a directory for the PyTorch project Enter the following commands: mkdir pytorch_pj cd pytorch_pj STEP 2: Launch Jupyter Notebook
AI Mathematics
Good morning! This is the Qualiteg Research Team. In this article, we explain how to express the cross-entropy computation using vectors and matrices. Chapter 8: Cross-Entropy and Vector Operations First of all, why would we want to express cross-entropy with vectors and matrices? The reason is that when you actually
![[AI-Driven New Business Creation] Why Divergent Thinking Is Essential to Idea Creation: The Qualiteg View](https://images.unsplash.com/photo-1525776658211-37bdb8a0b32b?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3wxMTc3M3wwfDF8c2VhcmNofDIzfHxicmFuY2hpbmclMjBvdXR8ZW58MHx8fHwxNzEzNzAxMTI2fDA&ixlib=rb-4.0.3&q=80&w=600)
AI-Business
When generating new business ideas, involve all internal stakeholders across diverse roles and backgrounds, not just your own department. This brings in multiple perspectives, makes feasible ideas more likely, strengthens communication early on, and leads to more concrete, effective solutions.
NumPy/PyTorch
Optimizing PyTorch Tensor Operations: Understanding and Resolving a Warning Message Hello! This is the Qualiteg Product Development Team. While working with PyTorch 1.13, we ran into the following warning message: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach(
![[ChatStream] Quickly Measuring Inference Speed Under Concurrent Requests](/content/images/size/w600/2024/05/--------------2-.png)
Hello, this is the Qualiteg Product Development Team. This time, we ran a quick measurement of RakutenAI-7B-Chat's performance when handling five simultaneous requests. Test Environment * GPU: A5000 * LLM: RakutenAI-7B-Chat AWQ-8 * Inference environment: ChatStream v0.7.2 * Number of concurrent requests: 5 Since a video is probably easier to
ChatStream Guide
[Appendix] Comparison of Penalties Penalty type Purpose How it is applied Example penalty Repetition Penalty Prevents specific tokens or phrases from being repeated. Applies a penalty to the log probabilities (logits) of all previously generated tokens. For example, if a token has already been generated, its log probability is divided
IT & AI Technology
Hello! This is the Qualiteg Tech Blog. [Updated July 2, 2024] Today we will cover the features and pricing of the Google Gemini 1.5 Pro / Gemini 1.5 Flash models, and how to generate text with them using Python. What is Google Gemini? Google Gemini is a generative AI
![[AI-Driven New Business Creation] Qualiteg's View on What to Do and What Not to Do in Idea Creation](https://images.unsplash.com/photo-1613143835030-b4a853181e32?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3wxMTc3M3wwfDF8c2VhcmNofDI0fHxkb24lMjd0fGVufDB8fHx8MTcxMzcwMTI4NHww&ixlib=rb-4.0.3&q=80&w=600)
AI-Business
At Qualiteg, we see idea creation as the foundation for new value. Embrace diverse perspectives, keep an open mind, use generative AI, and involve team members from different backgrounds. Prototype and validate early, avoid fixating on one idea, and stay flexible to choose the best option.
DIY GPU Machines
In this installment, we install CUDA and PyTorch. 4.2 Installing CUDA (+cuDNN) and PyTorch Together Installing PyTorch also pulls in CUDA and cuDNN for you, so let's take advantage of that. STEP 1: Generate the PyTorch installation command With all the groundwork finally in place, it'