Langchain Embedding connection timeout issue for document or large text

[Misto1997]

I have used Langchain's embed_query() and embed_document() methods and facing issue when these 2 methods calls _get_len_safe_embeddings() method.

1. On local machine both methods are working fine for short text, long text and for list.
2. But when I'm running the same from my k8's cluster embed_query() for short text is working fine but for long text it is failing and embed_document() method is also failing with below error:
   Exception in embedding creations HTTPSConnectionPool(host='openaipublic.blob.core.windows.net', port=443): Max retries exceeded with url: /encodings/cl100k_base.tiktoken (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f68dc6ca8b0>: Failed to establish a new connection: [Errno 110] Connection timed out'))

Im using Azure's OpenAI Deployment and I have whitelisted our Azure's endpoint hence rest API's are working fine.

My Issue query is, does Langchain calls some other OpenAI's API's like mentioned in error "host='openaipublic.blob.core.windows.net', port=443" which i should get whitelisted?

langchain version = 0.0.199
openai version = 0.28.1

[dosubot[bot]]

🤖

Hello @Misto1997! I'm here to assist you with your issue while we wait for a human maintainer. Just so you know, I'm a bot designed to help solve bugs, answer questions, and guide contributors. I'm here to make your journey with LangChain smoother. Let's dive into the issue you're facing!

Based on the information you've provided and the context from the LangChain repository, it seems like the connection timeout issue you're experiencing with the embed_query() and embed_document() methods on your k8's cluster is not related to whitelisting 'openaipublic.blob.core.windows.net'.

LangChain does not make calls to other OpenAI's APIs such as "host='openaipublic.blob.core.windows.net', port=443". The AzureOpenAIEmbeddings class in LangChain uses the Azure endpoint provided by the user or the environment variable AZURE_OPENAI_ENDPOINT. It does not make any reference to 'openaipublic.blob.core.windows.net'. You can confirm this in the AzureOpenAIEmbeddings class.

As for the _get_len_safe_embeddings() method, it is used to generate embeddings for a list of texts. It is designed to handle texts that are longer than the maximum context length allowed by the OpenAI API. It breaks down the tokens into chunks of a specified length (self.embedding_ctx_length), sends these chunks to the OpenAI API to generate embeddings, and then averages the embeddings for each text. You can find more details about this method in the OpenAIEmbeddings class.

Regarding the connection timeout issue, there was a similar issue reported in the LangChain repository titled OpenAIEmbeddings fails to connect. The suggested solution was to check the openai_proxy attribute and make sure it's either not set or set to a working proxy. The default value of the openai_proxy attribute in the OpenAIEmbeddings class in LangChain is None.

I hope this information helps. If you have any other questions or need further clarification, feel free to ask.

Sources

• OpenAIEmbeddings fails to connect
• libs/langchain/langchain/embeddings/azure_openai.py
• libs/langchain/langchain/embeddings/openai.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[Misto1997]

@dosu-bot

I tried using "AzureOpenAIEmbeddings" instead of "OpenAIEmbeddings" but both internally uses same OpenAi Public method.

on debugging, I found that langchain -> embeddings -> openai.py -> _get_len_safe_embeddings() method uses "tiktoken" package which makes use of "https://openaipublic.blob.core.windows.net/encodings/cl100k_base.tiktoken" this OpenAi public URL.

As an organisation using public URL is a breach of org policies that's why we were using Azure deployments but these implementations hinder use cases.

Do you have any suggestions on this?

[dosubot]

🤖

Based on the information you've provided, it seems that the issue arises from the use of a public OpenAI URL in the _get_len_safe_embeddings() method, which is against your organization's policies.

To resolve this, you can modify the _get_len_safe_embeddings() method to use a private URL instead of a public OpenAI URL. This can be achieved by setting the openai_api_base attribute of the OpenAIEmbeddings class to your private URL. This attribute is used to set the base URL for API requests.

Here is an example of how you can do this:

from langchain.embeddings.openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(
    openai_api_key="your-api-key",
    openai_api_base="https://your-private-url.com/",
)

In the above code, replace "your-api-key" with your actual OpenAI API key and "https://your-private-url.com/" with your actual private URL.

Please note that you need to ensure that your private URL is correctly set up to handle the requests made by the LangChain framework.

As for the _get_len_safe_embeddings() method, it is designed to generate embeddings for a list of texts. This method is particularly useful when dealing with long texts that exceed the maximum token limit of the OpenAI API. It tokenizes the input texts, splits them into chunks, generates embeddings for each chunk, and then calculates the average of the embeddings for each text.

I hope this helps! If you have any further questions or need more clarification, feel free to ask.

Sources

• libs/langchain/langchain/embeddings/openai.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[BrunoDesquesne]

I also have an error in a secured network without internet access. I don't undestand why the function "embedding_function" try to access to openaipublic.blob.core.windows.net

It would be dirty to discover public blob storage is used for our private embeddings !?

Does a human can help me ?

[jscott0918]

We're also experiencing this in a site without internet access and can't figure out how to point all endpoints through our api.

[Misto1997]

I'm not using any proxy and also mentioned thread: #12714 is not relevant as my error is not related to the proxy.

Note: embed_query() method is working fine for text length < embedding_ctx_length: int = 8191.
Only failing for long text and list which is calling method "_get_len_safe_embeddings" from this method

    def _embedding_func(self, text: str, *, engine: str) -> List[float]:
        """Call out to OpenAI's embedding endpoint."""
        # handle large input text
        if len(text) > self.embedding_ctx_length:
            return self._get_len_safe_embeddings([text], engine=engine)[0]
        else:
            if self.model.endswith("001"):
                # See: https://github.com/openai/openai-python/issues/418#issuecomment-1525939500
                # replace newlines, which can negatively affect performance.
                text = text.replace("\n", " ")
            return embed_with_retry(
                self,
                input=[text],
                **self._invocation_params,
            )[
                "data"
            ][0]["embedding"]