From 21ffed3f33d7919057e0d1c827fcf5b4709712aa Mon Sep 17 00:00:00 2001 From: Judd Date: Sat, 9 Dec 2023 18:10:38 +0800 Subject: [PATCH] Update README.md add notes on ChatLLM.cpp. --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index 93b2e10..5f35bba 100644 --- a/README.md +++ b/README.md @@ -346,6 +346,9 @@ python convert-hf-to-gguf.py --outfile --model-name dee ./quantize q4_0 ./main -m -n 128 -p ``` + +You can also try out other [ggml](https://github.com/ggerganov/ggml)-based inferencers, such as [ChatLLM.cpp](https://github.com/foldl/chatllm.cpp), as well. + #### GPTQ(exllamav2) `UPDATE:`[exllamav2](https://github.com/turboderp/exllamav2) has been able to support HuggingFace Tokenizer. Please pull the latest version and try out.