I’m currently working on activating conversation features in my Spring AI application but encountered a significant error. When I attempt to enable conversation functionality using ChatMemory, I receive the following error:

"*2024-11-05T10:57:17.441+01:00 ERROR 44072 --- [Bip.it] [oundedElastic-4] o.s.ai.chat.model.MessageAggregator : Aggregation Error

org.springframework.web.reactive.function.client.WebClientResponseException: 413 Payload Too Large from POST https://api.groq.com/openai/v1/chat/completions*"

This issue arises when I include the following code in RagService:

public RagService(ChatClient.Builder chatBuilder, VectorStore vectorStore) {

ChatMemory chatMemory = new InMemoryChatMemory();

this.chatClient = chatBuilder

.defaultAdvisors(

new MessageChatMemoryAdvisor(chatMemory),

new QuestionAnswerAdvisor(vectorStore),

new SimpleLoggerAdvisor())

.build();

}

Comment From: asaikali

How many messages are in the history before are getting this error message. I have created issue #1704 as a generalized version of what you are asking for.

@aniwange33 can you please us know if you got around your issues by reducing the amount of message in the chat memory, can you provide a sample reproducer of the issue.

Comment From: aniwange33

How many messages are in the history before are getting this error message. I have created issue #1704 as a generalized version of what you are asking for.

@aniwange33 can you please us know if you got around your issues by reducing the amount of message in the chat memory, can you provide a sample reproducer of the issue.

I couldn’t log the number of messages in the history without getting an error. However, when I used Filter.Expression, the error didn’t occur. Despite this, the conversation activation hasn’t been working smoothly, and sometimes the responses are inaccurate.

public RagService(ChatClient.Builder chatBuilder, VectorStore vectorStore, @Value("classpath:/system.md") Resource systemPrompt) { ChatMemory chatMemory = new InMemoryChatMemory(); var fb = new FilterExpressionBuilder(); Filter.Expression source = fb.eq("source", "FY25-MER-2.8-Indicator-Reference-Guide.pdf").build(); SearchRequest searchRequest = SearchRequest.defaults().withTopK(2).withFilterExpression(source); List advisors = List.of( new PromptChatMemoryAdvisor(chatMemory), new QuestionAnswerAdvisor(vectorStore, searchRequest), new SimpleLoggerAdvisor() ); this.chatClient = chatBuilder.defaultSystem(systemPrompt).defaultAdvisors(advisors).build(); }

public Flux streamChatWithConversationEnable(QueryRequest request) { log.info("message {}", request.getQuery()); Parser parser = Parser.builder().build(); HtmlRenderer renderer = HtmlRenderer.builder().build(); return Flux.defer(() -> chatClient.prompt() .user(request.getQuery()) .advisors(spec -> spec .param(CHAT_MEMORY_CONVERSATION_ID_KEY, request.getConversationId()) .param(CHAT_MEMORY_RETRIEVE_SIZE_KEY, 2) ) .stream().content() .bufferUntil(s -> s.endsWith(" ")) .map(list -> { StringBuilder contentBuilder = new StringBuilder(); list.forEach(contentBuilder::append); return contentBuilder.toString(); }) .map(content -> { String adjustedContent = content .replaceAll(" - ", "\n - ") .replaceAll("(?i)(\bNote:\b|\bWarning:\b|\bImportant:\b)", "$1") .replaceAll("(?i)\b(Note|Warning):\b", "$1:") .replaceAll("\s\n{2,}(?![^()]\))", "\n\n- "); Node document = parser.parse(adjustedContent); String render = renderer.render(document); BipChatResponse chatResponse = new BipChatResponse(); chatResponse.setConversationId(request.getConversationId()); chatResponse.setResponseContent(render); // Set plain text instead of HTML return chatResponse; }) ); }