Ideas on making the querying by the model, faster?

[frroossst]

We are experimenting with this internally at work, for some of our proprietary codebases and documentation, and we're really excited about the privacy focused nature of it. What can we (as non A.I. devs) do, to make the queries faster, they take about 1-2 minute(s) for us, not quite what we're looking for though, thanks!

[thekit]

maybe try reduce chunk overlap size and reduce the number of source documents the answer references
#251 (comment)

[maozdemir]

You can enable the GPU functionality.
The notebook below is for Linux and Google Colab, though nothing much is different in Windows:

https://github.com/maozdemir/privateGPT-colab