Bug description I am getting 0s for (promptTokens, completionTokens, and totalTokens) when trying to get the token consumption for a particular query.
This is happening in a simple RAG application backed by Spring AI 1.0.0-M2, PG vector database and spring-ai-openai-spring-boot-starter package.
I am setting up ChatClient object with advisors QuestionAnswerAdvisor(vectorStore, SearchRequest.defaults()) and MessageChatMemoryAdvisor(new InMemoryChatMemory())
I am getting ChatResponse object with EmptyUsage object inside, and thus zeros for promptTokens, completionTokens, and totalTokens.
Just dropped the version of Spring AI from M2 to M1 to test, and I am getting the token consumption details in M1.
Otherwise when I am commenting out QuestionAnswerAdvisor(vectorStore, SearchRequest.defaults()) than also I am getting the token consumption in the console.
As now I am getting Usage as part of ChatResponse instead of EmptyUsage.
Environment MS Windows 10, Spring AI version 1.0.0-M2, Java version 21, locally running postgres instance's vector store. Spring Boot version, 3.3.4.
Note: I am NOT using Docker compose.
Steps to reproduce Just setup a basic spring boot project with dependencies - spring-boot-starter-web, spring-ai-openai-spring-boot-starter, spring-ai-tika-document-reader, spring-ai-pgvector-store-spring-boot-starter, and for DB - postgresql, spring-boot-starter-jdbc.
Open 2 endpoints one for uploading PDF files and other to ask questions around those, a typical RAG flow. One can System.out.println ChatResponse object to see we get EmptyUsage object when using Spring AI M2 and Usage object when using Spring AI M1 instead.
Expected behavior
I guess even in M2 version as well the token consumption details should be provided in the ChatResponse object with Usage inside it instead of EmptyUsage.
Minimal Complete Reproducible example https://github.com/Reynolds045/simpleRag
follow the README.md in there.
Comment From: alexcheng1982
Can you please try the latest snapshot version of Spring AI and see if this problem still exists?
Comment From: dafriz
Fixed in https://github.com/spring-projects/spring-ai/issues/1537
Comment From: Reynolds045
Thanks, @alexcheng1982 and @dafriz it is working fine now with both versions M2 and M3.