Qualiteg - ChatStream Guide - Qualiteg Journal (Page 2)

ChatStream Guide

[ChatStream] Implementing the Web API Endpoint

Hello from the Product Development Department at Qualiteg Inc. In this article, we explain how to implement ChatStream as a FastAPI web API. Implementing the Endpoint To create a web endpoint for streaming chat at the URL path /chat_stream, call handle_chat_stream_request as shown below. That alone

ChatStream Guide

[ChatStream] Creation and Initialization

Hello from the Product Development Division at Qualiteg. In this article, we explain how to create and initialize ChatStream. The ChatStream class is the core class of the ChatStream package. It receives a FastAPI/Starlette Request and is responsible for sending a streaming response to the client while controlling the

ChatStream Guide

[ChatStream] Configuring the HTTP Session Middleware

Hello from the Product Development Department at Qualiteg. In this article, we explain how to set up a session middleware in ChatStream and make your web application stateful. ChatStream uses a proprietary session middleware developed by Qualiteg, which offers greater flexibility than the standard session middleware for Starlette. (Its approach

ChatStream Guide

[ChatStream] How to Load a Model

Hello, this is the Product Development Department at Qualiteg Inc. In this article, we explain the approach for loading Hugging Face models into ChatStream. Loading a Hugging Face Model Load the Hugging Face model using the method specified for each model. model_path = "togethercomputer/RedPajama-INCITE-Chat-3B-v1" device = "cuda&

ChatStream Guide

[ChatStream] Implementing ChatPrompt

What Is ChatPrompt? Hello, this is the Product Development Department at Qualiteg Inc. In this article, we walk through how to implement a ChatPrompt in concrete terms. ChatPrompt is a class for generating prompts for a pretrained language model (hereafter, "the model"). We call it a prompt class.

ChatStream Guide

[ChatStream] Quickstart

Hello from the Product Development Division at Qualiteg Inc. The summer heat just keeps going, doesn't it? Let's jump right in and build a real-time streaming chat server using ChatStream, which we announced yesterday. Installing the package First, install the ChatStream package. pip install chatstream Installing

ChatStream Guide

[ChatStream] ChatPrompt for mosaicml/mpt-7b-chat

Today we introduce a ChatPrompt for mosaicml/mpt-7b-chat. class ChatPromptMosaicmlMptChat(AbstractChatPrompt): """ mosaicml/mpt-7b-chat """ def __init__(self): super().__init__() # Call the initialization of the base class self.set_requester("<human>") self.set_responder("<bot>") self.set_prefix_

ChatStream Guide

[ChatStream] ChatPrompt for matsuo-lab/weblab-10b-instruction-sft

We have released a ChatPrompt class for matsuo-lab/weblab-10b-instruction-sft. It is included in the latest version of ChatStream, but you can also use the code below. from chatstream import AbstractChatPrompt from chatstream.chat_prompt.prompt_ttl import PromptTTL class ChatPromptMatsuoLabJpGptNeoxInstSft(AbstractChatPrompt): def __init__(self): super().__init__() # Call the initialization of

News

[ChatStream] Announcing the ChatStream α1 (Preview) Release

Qualiteg Inc. has begun offering the alpha preview release of ChatStream™, our toolkit for developing LLM applications. ChatStream™ is a Python library that makes it easy to build high-quality, commercial-grade LLM applications. It consists of a web front end (chat UI) and inference server software, so you can deliver a