Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom Functions & Retrival of Documents, Notes... with PrivateGPT or Anything LLM #202

Open
rgsaura opened this issue Apr 23, 2024 · 5 comments

Comments

@rgsaura
Copy link

rgsaura commented Apr 23, 2024

Hi! I'm trying to get a chatbot that can do custom functions and get context from Notes. PrivateGPT & Anything LLM seem like two good options as they let you upload data to a vector database and then retrive it.

I need help setting something with these characteristics in place. If you like this idea please say something on the comments or up vote this pls! Help make this possible 🙏🏼

@jleinenbach
Copy link

jleinenbach commented Apr 24, 2024

PrivateGPT may work out of the box as it follows the OpenAI API standard, but without the custom API features.

The project provides an API offering all the primitives required to build private, context-aware AI applications. It follows and extends the OpenAI API standard, and supports both normal and streaming responses.

@rgsaura
Copy link
Author

rgsaura commented Apr 24, 2024

I tried PRIVATEGPT with all possible combinations without success. I cant get another thing other than "detail": "Not Found", so it's reaching the server, but in an incompatible way or I'm doing something wrong.... 😭

Any help / ideas would be apreciated! @jekalmin @jleinenbach 😄 🚀

Screenshot 2024-04-24 at 12 14 55
Screenshot 2024-04-24 at 12 14 29
Screenshot 2024-04-24 at 12 14 18
Screenshot 2024-04-24 at 12 14 12
Screenshot 2024-04-24 at 12 15 17

@jleinenbach
Copy link

When I search for the error on Google, I find several possible causes. One possibility could be that the base URL is not configured correctly.
Did you try http://100.126.37.110:8001/api/v1 as base URL?

@rgsaura
Copy link
Author

rgsaura commented Apr 24, 2024

As I understand it combines this two fields. but I also did try several combinations without success. Apreciate quick response!

Screenshot 2024-04-24 at 12 47 37

@jleinenbach
Copy link

Even if you check "Skip Authentication", you still have to enter an API key value. This is a bit confusing.
So as you set it to "-", the api_key variable may still be filled with this value and you need to trust the software not to set an API key.

According to the error message, your error happens in helpers.py: "validate authentication".

This is the code:

async def validate_authentication(
    hass: HomeAssistant,
    api_key: str,
    base_url: str,
    api_version: str,
    organization: str = None,
    skip_authentication=False,
) -> None:
    if skip_authentication:
        return

    if is_azure(base_url):
        client = AsyncAzureOpenAI(
            api_key=api_key,
            azure_endpoint=base_url,
            api_version=api_version,
            organization=organization,
        )
    else:
        client = AsyncOpenAI(
            api_key=api_key, base_url=base_url, organization=organization
        )

    await client.models.list(timeout=10)

This function should do nothing if you check "Skip Authentication", so I guess you didn't check it it when this exception was logged.
What I can see is that if it's not Azure, the api_version seems to be igored.

At this point, I'd use this base URL:
http://100.126.37.110:8001/api/v1

On the first look, I didn't find a way how to create an API key as I would try it with a valid API key next:
I guess you installed an OpenAI Compatible Server as an LLM Backend:
https://docs.privategpt.dev/manual/advanced-setup/llm-backends
https://docs.vllm.ai/en/latest/getting_started/quickstart.html#openai-compatible-server

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants