Hello from the Product Development Department at Qualiteg.

In this article, we explain how to set up a session middleware in ChatStream and make your web application stateful.

ChatStream uses a proprietary session middleware developed by Qualiteg, which offers greater flexibility than the standard session middleware for Starlette. (Its approach is close to session management in Java Servlets.)

Session Middleware

To hold a multi-round conversation in a web chat within an open browser, the ChatPrompt (conversation history) must be updated across multiple turns of the conversation.

By default, ChatStream uses HTTP sessions to make the web application stateful, so that the ChatPrompt can be retained while the browser is open.

To use HTTP sessions, register the FastAPI middleware as shown below.

from fastersession import FasterSessionMiddleware, MemoryStore
app.add_middleware(FasterSessionMiddleware,
                   secret_key="your-session-secret-key",
                   store=MemoryStore(),
                   http_only=True,
                   secure=True,
                   )

Internal Processing

In this default implementation, a session ID is generated, signed, and then stored in a cookie.

The cookie is retained only while the browser is open, and it cannot be accessed from front-end JavaScript.

Other Ways to Persist Conversation History

In the default implementation, the ChatPrompt lives in the session. Session information is managed in memory on the server side, and the session lasts only while the browser is open.

When building a full-fledged chat server, it is common to authenticate users and store (persist) the ChatPrompt associated with each user in a database.

To do this, you can implement a custom request handler and manage and persist the ChatPrompt within that request handler.

Related: CORS Middleware

CORS Middleware