Is RAG (document grounding) feature integrated in Spring AI?

[zxcvboy]

Ask the Question

Dear colleague.

Is RAG feature supported by ai-sdk-java? If not, i there any roadmap? 

Thanks a lot.

[jjtang1985]

Hi @zxcvboy ,

Could you please share more details about your use cases?

• using the SharePoint with pipeline API
• adding embeddings with vector API

We planned to support the grounding features early next year.

[zxcvboy]

Hi. @jjtang1985

I want to retrieve some results from my vector engine and send those results with my prompt to LLM.
Below is my sample code with Spring AI. I just wonder if this feature can be achieved by ai-sdk-java or not.
------------ Spring AI sample -------------------------

private final VectorStore vectorStore; // Spring AI自動配置
private final ChatClient chatClient; // 使用Config類別建立好Bean,這裡可以直接綁定

public String search2(String query) throws IOException {

	String result = chatClient.prompt().advisors(
			new QuestionAnswerAdvisor(vectorStore) // RAG增強器
	).user(query).call().content();

	return result;
}

[MatKuhr]

Yes, you can achieve the same RAG flow with the AI SDK. However, currently you have to implement the embedding calls and vector database retrieval manually. We have it on the roadmap to provide an integration for Spring AI, but this isn't completed yet.

You can refer to the sample code on how to compute an embedding.

[zxcvboy]

Hi. @MatKuhr.

Appreciate for your reply. Is there a specific roadmap for integration with Spring AI? RAG is the most popular AI scenarios for SAP customers. Is it possible to provide a sample or scenario for a RAG use case?

Thanks a lot.

[MatKuhr]

Yes, we have actually just released the first iteration of support for Spring AI with version 1.2.0. You can find the documentation here. Currently, that is limited to the OrchestrationClient, which itself doesn't support embeddings at the moment, but can of course be used together with a vector store. We haven't tested if the current state of the integration already allows for the QuestionAnswerAdvisor to work around it, but feel free to try it out and let us know 😄

Alternatively, one can use the grounding capabilities of orchestration, if you don't want to implement the full RAG flow manually. If that is interesting for you, you can learn more here.

[newtork]

(update)

We planned to support the grounding features early next year.

FYI, we released 1.4.0 last Friday. It provides initial support for Document Grounding services. Stable convenience will follow soon.

The Spring AI integration for grounding is still waiting for implementation, but it'll be done in the following weeks hopefully.

[CharlesDuboisSAP]

Hello @zxcvboy,

Can you use the Orchestration grounding service, that uses the central HANA database, or, do you have to use your own HANA database?

[zxcvboy]

Hi. @CharlesDuboisSAP

I tried to leverage grounding capability of Orchestration, however I didn`t find out any document about how to add Vector DB into a resource group. So I am stuck right now.

[Jonas-Isr]

Hi @zxcvboy,

you can find documentation on how to set up the Orchestration vector API here. The steps outlined there are basically:

1. Create a resource group for grounding
2. Preparing Data Using the Vector API
3. Configure Grounding Module

Does that help?
Best,
Jonas

[MatKuhr]

Just as additional hint: In case you want to keep using the default resource group, you can also update that using PATCH with

{
	"labels": [   
    {
      "key": "ext.ai.sap.com/document-grounding", 
      "value": "true"
    }
  ]
}

[zxcvboy]

Hi @zxcvboy,

you can find documentation on how to set up the Orchestration vector API here. The steps outlined there are basically:

1. Create a resource group for grounding
2. Preparing Data Using the Vector API
3. Configure Grounding Module

Does that help? Best, Jonas

Hi. @Jonas-Isr .

Is it possible to do it with AI launchpad? Thanks a lot. 

[newtork]

Can you please make your question more specific? How is that related to AI SDK?