Empty Embeddings propertie when returning from Indexing a document #21119 #21121
-
Checked other resources
Commit to Help
Example Codeembeddings = AzureOpenAIEmbeddings(
model="text-embedding-3-small",
azure_deployment="text-embedding-3-small",
openai_api_version="2024-02-01",
)
vectorstore = Chroma(
collection_name="cv_collection",
embedding_function=embeddings,
)
st.session_state.record_manager = SQLRecordManager(
db_url="sqlite:///:memory:",
namespace="chroma/cv_collection",
)
def add_document_to_vectorStore(vectorStore, document, record_manager):
# Index the new document in the vector store with incremental cleanup
index(
[document], # Pass a list of documents
record_manager,
vectorStore,
cleanup="incremental",
source_id_key="source",
)DescriptionHi, im trying to embed a document in a Chroma vector store, my issue is that when i call the Index function it does index it but it adds it as documents and not their corresponding embeddings, also when i call the function i can see the Embeddings request to openAI going out, but then the dump returns and "embeddings": null followed by the whole document like this: {
"ids": [
"a68dbda2-06bf-50df-b460-f2d1cd8dc8c0"
],
"embeddings": null,
"metadatas": [
{
"departamento": "Artigas",
"idioma": "Inglés, Portugues",
"nivel_educativo": "TERCERIA_SUPERIOR",
"source": "Laura Texeira._.pdf"
}
],
"documents": [
"Text"
],
"uris": null,
"data": null
}I have tried from creating a small proyect with only this to already reading all the chromadb and langchain indexing documentation, to changing from embeddings to the client for AzureOpenAI and the embeddings are being computed perfectly everytime, i also can see the request coming back as 200 OK, so i dont really know where else to look outside de Index function and thats why i think is a bug. System InfoIm in a docker container with the latest versions of this packages:
and a base image of python:3.9-slim-buster |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 2 replies
-
|
It looks like the issue with
By focusing on these areas, you should be able to narrow down the cause of the issue. If the problem persists, providing more details about the
|
Beta Was this translation helpful? Give feedback.
-
|
@dosu i have already tried calling this exact configuration outside the Index function like this: embeddings = AzureOpenAIEmbeddings(
model="text-embedding-3-small",
azure_deployment="text-embedding-3-small",
openai_api_version="2024-02-01",
)
document_embedding = embeddings.embed_query(document.page_content)
logging.info(f"Embedding results: {document_embedding[:5]}")And its working perfectly, like i said again, i can even see the call to the embeddings api returning 200 when i call the Index function, but it still returns with embeddings as null. And i dont manage the logic that stores the embeddings, thats suppose to be done by the Index function as far as im concerned |
Beta Was this translation helpful? Give feedback.
-
|
Since you've confirmed that embedding generation works correctly outside the Given that you're not directly managing the logic for storing embeddings and that the embedding generation is confirmed to be working, the next steps involve looking into how the To troubleshoot further, consider the following steps:
By focusing on how embeddings are handled and stored after generation, especially in relation to the
|
Beta Was this translation helpful? Give feedback.
Since you've confirmed that embedding generation works correctly outside the
indexfunction and the API call to AzureOpenAIEmbeddings returns a 200 OK, the issue likely lies in how these embeddings are handled or stored by theindexfunction within the Chroma vector store. Theindexfunction is designed to automate the process of storing documents and their embeddings, but if embeddings are coming back as null, it suggests a disconnect between the generation of embeddings and their storage.Given that you're not directly managing the logic for storing embeddings and that the embedding generation is confirmed to be working, the next steps involve looking into how the
indexfunction integr…