[ChatStream] ChatPrompt for rinna/nekomata-14b-instruction

In this article we introduce a ChatPrompt for rinna/nekomata-14b-instruction, which was released on December 21, 2023.

The nekomata series is based on Qwen and has a vocabulary size of 152,000, considerably larger than earlier models, so we have high expectations for its Japanese language capabilities.

This time we created a ChatPrompt for using the instruction-tuned model in ChatStream's chat mode.

The underlying design philosophy is to map one task to one ChatPrompt. Put simply, each ChatPrompt is specialized for a single job.

For example, the ChatPrompt shown here is specialized for the task of "translation."

With this, the "input" and "output" for the translation task are handled through the chat interface.

from chatstream import AbstractChatPrompt
from chatstream.chat_prompt.prompt_ttl import PromptTTL


class ChatPromptRinnaNekomata(AbstractChatPrompt):

    def __init__(self):
        super().__init__()

        # The Japanese prompt template below is part of the model's specification, so it is kept as is.
        # system_message: "The following is a combination of an instruction describing a task and an input with context. Write a response that appropriately satisfies the request."
        # instruction_message: "Translate the following Japanese into English."
        system_message = "以下は、タスクを説明する指示と、文脈のある入力の組み合わせです。要求を適切に満たす応答を書きなさい。"
        instruction_message = "次の日本語を英語に翻訳してください。"
        # "### 指示:" = "### Instruction:"
        self.set_system(f"{system_message}\n\n### 指示:\n{instruction_message}")
        self.set_requester("入力")  # "Input"
        self.set_responder("応答")  # "Response"
        self.set_prompt_ttl(PromptTTL.SINGLE_TURN)

    def get_stop_strs(self):
        if not self.chat_mode:
            return None
        return ["<|endoftext|>"]

    def get_replacement_when_input(self):
        return None

    def get_replacement_when_output(self):
        return None

    def create_prompt(self, opts={}):
        if not self.chat_mode:
            return self.get_requester_last_msg()

        ret = self.system + "\n\n"
        for chat_content in self.get_contents(opts):

            chat_content_role = chat_content.get_role()
            chat_content_message = chat_content.get_message()

            if chat_content_role:

                if chat_content_message:
                    merged_message = f"### {chat_content_role}:\n" + chat_content_message + "\n\n"
                else:
                    merged_message = f"### {chat_content_role}:\n"

                ret += merged_message

        return ret