Is RAG feature supported by ai-sdk-java?

[zxcvboy]

Ask the Question

Dear colleague.

Is RAG feature supported by ai-sdk-java? If not, i there any roadmap? 

Thanks a lot.

[jjtang1985]

Hi @zxcvboy ,

Could you please share more details about your use cases?

• using the SharePoint with pipeline API
• adding embeddings with vector API

We planned to support the grounding features early next year.

[zxcvboy]

Hi. @jjtang1985

I want to retrieve some results from my vector engine and send those results with my prompt to LLM.
Below is my sample code with Spring AI. I just wonder if this feature can be achieved by ai-sdk-java or not.
------------ Spring AI sample -------------------------

private final VectorStore vectorStore; // Spring AI自動配置
private final ChatClient chatClient; // 使用Config類別建立好Bean,這裡可以直接綁定

public String search2(String query) throws IOException {

	String result = chatClient.prompt().advisors(
			new QuestionAnswerAdvisor(vectorStore) // RAG增強器
	).user(query).call().content();

	return result;
}

[MatKuhr]

Yes, you can achieve the same RAG flow with the AI SDK. However, currently you have to implement the embedding calls and vector database retrieval manually. We have it on the roadmap to provide an integration for Spring AI, but this isn't completed yet.

You can refer to the sample code on how to compute an embedding.

[zxcvboy]

Hi. @MatKuhr.

Appreciate for your reply. Is there a specific roadmap for integration with Spring AI? RAG is the most popular AI scenarios for SAP customers. Is it possible to provide a sample or scenario for a RAG use case?

Thanks a lot.

[MatKuhr]

Yes, we have actually just released the first iteration of support for Spring AI with version 1.2.0. You can find the documentation here. Currently, that is limited to the OrchestrationClient, which itself doesn't support embeddings at the moment, but can of course be used together with a vector store. We haven't tested if the current state of the integration already allows for the QuestionAnswerAdvisor to work around it, but feel free to try it out and let us know 😄

Alternatively, one can use the grounding capabilities of orchestration, if you don't want to implement the full RAG flow manually. If that is interesting for you, you can learn more here.