Answers become "acceptable"...

[fenixlam]

I keep testing the privateGPT for several weeks with different versions, I can say that privateGPT's accuracy is very low.
It looks like it can only read the last document, and mostly it cannot get the correct answer.

Recently I watch youtube and found a localGPT project, which is similar to privateGPT.
(I can only use CPU to run the projects, so... I can only keep going on privateGPT )
The main point is, I copy localGPT's ingest.py, use hkunlp/instructor-xl as the embedding model to generate chomo database files.
(which requires a very long time to generate that database file...)
Then I use WizardLM-30B-Uncensored.ggmlv3.q4_0 as the LlamaCpp model.
(which takes a very long time to get the answer in qa...)
It gets acceptable answers. I can say that is a great improved.

So I suspect the accuracy problem is coming from the fastest embedding model all-MiniLM-L12-v2 and all-MiniLM-L6-v2.
Because I have tested it with WizardLM-30B-Uncensored.ggmlv3.q4_0, the accuracy is still very low...

[77bgh]

in my case , when running with this config
MODEL_PATH=models/ggml-gpt4all-j-v1.3-groovy.bin
EMBEDDINGS_MODEL_NAME=all-MiniLM-L12-v2

the answer is very low comparing to
MODEL_PATH=models/ggml-gpt4all-j-v1.3-groovy.bin
EMBEDDINGS_MODEL_NAME=all-MiniLM-L6-v2

however please consider the answer reflects the ingested documents' content

[fenixlam]

All Test are using the same model: ggml-gpt4all-j-v1.3-groovy (GPT4All)

L6:
Enter a query: what is Tori's age?
The given text does not provide any information about Tori's age.

L12:
Enter a query: what is Tori's age?
Tori's age is not specified in the given context.

instructor_x1:
Enter a query: what is Tori's age?
Tori is an 8-year-old girl. #correct....

And I want to said... gpt4all cannot setup temperature=0.
I suspect its default value is not 0, so it may return fake answer.
LlamaCpp can setup temperature=0, so most of my tests are basic on LlamaCpp.
One of the test methods is to ask them summarize your story.
The answer's accuracy will not be affected by the wrong question and strange document content.

To use instructor_x1, you have to git clone localGPT. Copy your documents to localGPT and ingest the DB files.
Then copy the db files to privatGPT project. These actions can be run on CPU only.
instructor_x1 code in privateGPT.py:

#original embeedings engine:
from langchain.embeddings import HuggingFaceEmbeddings, HuggingFaceInstructEmbeddings
    if persist_directory=='db_instructor':   
        print('using db_instructor')
        embeddings = HuggingFaceInstructEmbeddings(model_name="hkunlp/instructor-xl", model_kwargs={"device": 'cpu'})
    else:
        print('using L6 or L12')
        embeddings = HuggingFaceEmbeddings(model_name=embeddings_model_name)

[pseudotensor]

Yes, I also found GPT4All gpt-j models etc. are quite poor, sometimes even natively giving no output.

I recommend better than Vicuna, instead use WizardLM uncensored models from TheBloke for CPU and his -HF models, and use our prompt_type that matches in h2oGPT: https://github.com/h2oai/h2ogpt#windows-1011

[3x3cut0r]

It looks like it can only read the last document, and mostly it cannot get the correct answer.

Why is that? Why can it not read over all documents? Can i do something about that?