[ChatStream] Announcing the ChatStream α1 (Preview) Release

Qualiteg Inc. has begun offering the alpha preview release of ChatStream™, our toolkit for developing LLM applications.

ChatStream™ is a Python library that makes it easy to build high-quality, commercial-grade LLM applications.

It consists of a web front end (chat UI) and inference server software, so you can deliver a complete LLM application with ChatStream™ alone, without any additional software development.

For the LLM at its core, you can use open-source models available on HuggingFace and similar platforms. (Connecting to APIs such as ChatGPT is also possible.)

As of September 2023, it is available as a Python library under the AGPL and a commercial license.

Please try the online demo at the site below.

https://chatstream.net

You can build applications like this demo with almost no code.

ChatStream is designed for serious commercial use and offers the following features.

Key features of ChatStream

1. A chat UI that delivers a great user experience

   1. A ChatGPT-like interface that users find intuitive
   2. Flexible chat UI design that supports multimodal input

2. Build high-performance, high-quality chat apps with a short turnaround

   1. Install with a single command
   2. Low-code: build a full-fledged LLM chat with just a few lines of code

3. Designed for heavy load

   1. Asynchronous text generation keeps responses stable even with many concurrent users
   2. Multi-node configuration makes scaling out easy

4. Flexible runtime environment (no dependence on any specific platform)

   1. Verified with major LLMs such as Llama, RedPajama-INCITE, and Rinna
   2. Designed to accommodate a variety of user authentication mechanisms
   3. Runs on-premises and on various cloud platforms
   4. Inference runs on CPUs and NVIDIA GPUs (multi-GPU supported)

5. Accelerates development of high-UX apps

   1. A rich set of customizable UI presets
   2. Development support features that help refine LLM output

The system requirements are as follows.
(It runs in a standard Python + NVIDIA GPU environment.)

System requirements

• Python

  • Python 3.10 or later
  • PyTorch

• GPU

  • NVIDIA GPU with CUDA 11.7 support

• Load balancing

  • Multi-GPU scale-out
  • Multi-node scale-out
  • Load control via request queuing

• OS

  • Linux
  • Windows Server

• API

  • Expose LLMs as APIs

• Encryption (TLS/SSL)

  • Supported

• User authentication

  • OAuth authentication
  • Email authentication
  • (Custom authentication can also be supported)

Support

• Extensive Japanese-language documentation
• Attentive support
  • Because ChatStream was developed from scratch in-house, hands-on support is provided in English and Japanese by engineers who know every corner of it
  • Support is available during Japanese working hours, with no time-zone lag

In upcoming posts, we will introduce ChatStream in more detail from both business and technical perspectives.

Qualiteg Inc. provides consulting and support for solving your business challenges with generative AI, building LLM applications on the ChatStream platform, and running PoCs.

For inquiries or requests, please contact your Qualiteg sales representative or reach us via the contact form below.

https://qualiteg.com/contact