[Request] : Question about Limiting Document to Search

[pythonmanGo]

🥰 需求描述

I'm building a RAG (Retriever-Augmented Generation) knowledge base, and I've encountered an issue where, no matter what prompt I use, the system always searches for the top 5 most related documents, rather than searching through all documents in the knowledge base.

Even when I specify a document's name, the system still associates it with the top 5 documents based on keyword relevance, rather than directly querying the specified document.

My question is : How to use prompts to ask questions directly about specific documents, or to require the system to search through all documents for relevant information, instead of limiting the search to the top 5 related ones?

🧐 解决方案

Use prompt to ask questions directly about specific documents, or to require the system to search through all documents for relevant information, instead of limiting the search to the top 5 related ones

📝 补充信息

No response

[lobehubbot]

👀 @pythonmanGo

Thank you for raising an issue. We will investigate into the matter and get back to you as soon as possible.
Please make sure you have given us as much context as possible.
非常感谢您提交 issue。我们会尽快调查此事，并尽快回复您。 请确保您已经提供了尽可能多的背景信息。

[dosubot[bot]]

To address the issue of limiting the search to the top 5 documents, you can modify the semantic search function to expand the number of documents it retrieves. The current implementation limits the results to 5 entries for chat-specific searches. You can adjust this limit by changing the limit parameter in the query.

Here's how you can modify the code to search through more documents:

1. Locate the semantic search function in the chunk.ts file.
2. Adjust the limit parameter in the query to a higher number to allow more documents to be retrieved.

For example, change:

.limit(5);

to:

.limit(30); // or any other number you prefer

This change will allow the system to consider more documents during the search process, rather than restricting it to the top 5 based on similarity ^[1].

Additionally, if you want to query specific documents by name or ID, you can use the fileIds parameter to filter the search to those specific documents. This way, you can directly query the specified document without it being associated with the top 5 documents based on keyword relevance ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[arvinxx]

currently we limit the topN number to 5. So the rag quality is not so good. We will make a more flexable ui to make you control the topN of the search result.