Support for privateGPT, including support for RAG functions like rspecting context and parsing of sources

[Aquan1412]

Hi, as suggested in our recent discussion #305, here's a PR that adds support for interacting with privateGPT. Its based on the openai-backend, with some additions. The focus was to enable the use of two specific key words, as described in the API reference:

1. use_context: directs the LLM to use the context of documents that have been "ingested"
2. include_sources: directs the LLM to also return information about the sources of the context (so far: file name and page number, if appropriate).

I changed the code of gptel-openai.elto send the additional keywords to the llm server, and to correctly parse the answer if sources are provided.

I tested the code with different queries, and so far it works for my use case. However, there is at least one missing feature: currently, the two new keywords use_context and include_sources are hardcoded to t. It would probably be better to make them configurable when creating the backend. Unfortunately, I'm not sure how to do that correctly.

[karthink]

@Aquan1412 Thanks for the PR.

I had a look at the code, It uses free/global variables and uses some cl-lib functions that aren't declared at compile-time. I can clean up the code but I need some sample privategpt output to be sure I don't break anything.

Could you run (setq gptel-log-level 'info), use privategpt so it generates some sources (more than one), and paste the results of the *gptel-log* buffer here?

[Aquan1412]

Thanks for cleaning up the code, I'm unfortunately not very experienced writing elisp code.

I ran the command you asked for, however, the output is quite large, which is why I appended it as a file to this comment.

The user query and the expected response are as follows:

*** What are the Navier-Stokes equations?

The Navier-Stokes equations are a set of partial differential equations that describe the conservation of mass, momentum, and energy for a viscous fluid. They are given by equation (2.1) for mass conservation, equation (2.2) for momentum conservation, and equation (2.3) for energy conservation in compressible form. The pressure p, density ρ, velocity components ui, vi, wi, specific internal energy e, specific enthalpy h, temperature T, effective stress tensor τij, and convective heat flux qj are involved in these equations. For gases, the ideal gas law (2.4) relates pressure p to density ρ and specific volume RT/γ, where R is the gas constant and γ is the ratio of specific heats. The specific internal energy e and enthalpy h can be expressed as functions of temperature T using equations (2.5).

Sources:

• Beneddine-2017-Characterization_of_unsteady_flow_behavior_by_linear_stability_analysis.pdf (page 62)
• Iorio-2015-Global_Stability_Analysis_of_Turbulent_Transonic_Flows_on_Airfoil_geometries.pdf (page 33)

gptel-privategpt-output.txt

[karthink]

From what I can see, you're not actually buffering the sources from each response. persistent-sources in your code is reset with every chunk, so when it's finally processed it's set to the sources included in the last-but-one response chunk. Is this intentional?

• Can each chunk have different sources,
• or is the list of sources the same across all chunks,
• or is the list of sources provided with each chunk an accumulation of the sources used so far?

[Aquan1412]

As far as I'm aware,the list of sources is the same across all chunks. I didn't find an explicit confirmation in the privateGPT API description, but in all my tests it was like that.
Therefore I only always reset the chunk, to avoid parsing the list of sources for each chunk.

[karthink]

Wow, that's really wasteful API design. It's sending about 200-1000x as much JSON as required then.

…

On Sun, May 19, 2024, 1:43 AM Aquan1412 ***@***.***> wrote: As far as I'm aware,the list of sources is the same across all chunks. I didn't find an explicit confirmation in the privateGPT API description, but in all my tests it was like that. Therefore I only always reset the chunk, to avoid parsing the list of sources for each chunk. — Reply to this email directly, view it on GitHub <#312 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACBVOLFKQJT7U5NVMPR7YQ3ZDBQ4JAVCNFSM6AAAAABHSVGPXWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJZGE2TIOJRGI> . You are receiving this because you commented.Message ID: ***@***.***>

[karthink]

I cleaned up the code and pushed a couple of commits to your branch. I also made "use_context" and "include_sources" configurable. Please check if this introduced any bugs, as I can't test it.

[Aquan1412]

Great, thanks! Unfortunately, I'm currently travelling and won't have access to my computer for the next two weeks. But afterwards I'll test it as soon as possible.