encounter KeyError when using similarity_search_with_relevance_scores

[yuning1990]

Has anyone got the same error?

After I merged a lot index vector, like about 22074 index keys, I wanna retrieve relevent documents using function like retriever.vectorstore.similarity_search_with_relevance_scores(last_query, **search_kwargs), I got the following error msgs:

···
len(db.docstore._dict) --- 22074
last_query--- 【one secret query】

Traceback (most recent call last):
File "/check_large_num_problem.py", line 101, in
query_test()
File "/check_large_num_problem.py", line 94, in query_test
relevant_documents = retriever.vectorstore.similarity_search_with_relevance_scores(last_query, **search_kwargs)
File "/anaconda3/envs/py310/lib/python3.10/site-packages/langchain_core/vectorstores.py", line 306, in similarity_search_with_relevance_scores
docs_and_similarities = self._similarity_search_with_relevance_scores(
File "/anaconda3/envs/py310/lib/python3.10/site-packages/langchain/vectorstores/faiss.py", line 1125, in _similarity_search_with_relevance_scores
docs_and_scores = self.similarity_search_with_score(
File "/anaconda3/envs/py310/lib/python3.10/site-packages/langchain/vectorstores/faiss.py", line 391, in similarity_search_with_score
docs = self.similarity_search_with_score_by_vector(
File "/anaconda3/envs/py310/lib/python3.10/site-packages/langchain/vectorstores/faiss.py", line 304, in similarity_search_with_score_by_vector
_id = self.index_to_docstore_id[i]
KeyError: 22109
···``

[Prateek-d3v]

Did you get any solution for this error? @yuning1990

[yuning1990]

I indexed some articles, and merged them, I found two of them vectors were indexed wrong, I deleted them, then things seemed get right