llama-cpp-python 回答混乱

[ovo-Tim]

我的代码:

from llama_cpp import Llama

PROMPT = '''
            你是一名翻译家，请将用户输入的内容翻译成中文，你的翻译应该尽可能准确，不要编造，注意联系上下文。一些专用术语请在中文后面打括号附上原文。并在适当的位置添加一些符合中国人习惯的译注，例如将非公制单位转换成公制单位。
            以下是一些翻译示例：
            原文：Let's create a cube with a width of 1 foot in OpenCASCADE.
            翻译: 让我们在 OpenCASCADE 当中创建一个宽为 1英尺(译注:1英尺≈30.48厘米) 的正方体吧。

            原文：The algorithm is like a beautiful symphony, each line of code flows like a rhythm. May this book gently ring in your mind, leaving a unique and profound melody.
            翻译：算法犹如美妙的交响乐，每一行代码都像韵律般流淌。愿这本书在你的脑海中轻轻响起，留下独特而深刻的旋律。
        '''

llm = Llama(model_path="/home/tim/llama/chinese-alpaca-2-7b/ggml-model-q4_0.gguf", verbose=False, chat_format='', n_gpu_layers=32)
llm(PROMPT)['choices'][0]['text']

while True:
    inp = input("输入:")
    # i.content = llm(f"原文: { i.content } 翻译:", stop=["原文:", "\n"], max_tokens=32)['choices'][0]['text']
    print("输出：", llm(inp, max_tokens=32, top_k=40, top_p=0.9, repeat_penalty=1.1)['choices'][0]['text'])

运行结果：

tim@tim-PC:~/llama$ /home/tim/.pyenv/versions/3.10.13/bin/python /home/tim/llama/test.py
输入:At compile time many possible concurrent programming problems are pointed out and the Rust compiler is so robust that it normally finds errors not found by other language compilers.
输出： 一切都在编译期间检查，如果问题被发现，则会阻止构建。
Rust 中的错误类型有很多种：
1. Syntax errors: 
输入:The algorithm is like a beautiful symphony, each line of code flows like a rhythm. May this book gently ring in your mind, leaving a unique and profound melody.
输出： 
12. A novel can be many things: it can be an escape from reality, a journey through time and space, a lesson on humanity

作为对比，使用 llama.cpp 同样提示词：

[INST] <<SYS>>
你是一名翻译家，请将用户输入的内容翻译成中文，你的翻译应该尽可能准确，不要编造，注意联系上下文。一些专用术语请在中文后面打括号附上原文。并在适当的位置添加一些符合中国人习惯的译注，例如将非公制单位转换成公制单位。            以下是一些翻译示例：            原文：Let's create a cube with a width of 1 foot in OpenCASCADE.
            翻译: 让我们在 OpenCASCADE 当中创建一个宽为 1英尺(译注:1英尺≈30.48厘米) 的正方体吧。<</SYS>>

 [/INST] 以下是一些翻译示例：原文：Let's create a cube with a width of 1 foot in OpenCASCADE.
翻译：让我们在OpenCASCADE中创建一个宽为1英尺的正方体吧。 [INST] At compile time many possible concurrent programming problems are pointed out and the Rust compiler is so robust that it normally finds errors not found by other language compilers.
 [/INST] 在编译时，许多可能的并发编程问题都会被指出来。Rust编译器通常会发现其他语言编译器无法找到的问题。 [INST] And basically cargo is the Rust package manager and build system and you can use it to download libraries and build them and do countless other different things which we will cover as we continue.
 [/INST] 基本上，cargo是Rust的包管理器和构建系统。您可以使用它下载库并对其进行编译以及完成无数其他不同的事情（我们将在后续内容中涵盖）。 [INST] In the description is a link to all the code along with the cheat sheets all on github.
 [/INST] 在描述中包含一个链接，指向所有代码和提示书都在GitHub上。 [INST] And basically the reason we're going to use Rust and why Rust is important is it provides a high performance that is similar to C++ and is a systems programming language that is excellent at reducing memory related errors.
 [/INST] 基本上，我们选择使用Rust并重视的原因是它提供了高性能（类似于C++）并且是一个用于系统编程的优秀语言，能够减少内存相关的错误。 [INST] The algorithm is like a beautiful symphony, each line of code flows like a rhythm. May this book gently ring in your mind, leaving a unique and profound melody.
 [/INST] 这个算法就像一首美妙的交响乐，每一行代码都像节奏一样流动。愿这本书在您的脑海中轻轻响起，留下独特的和深远的旋律。