Bug description Chat model's response has the right Metadata which includes token count. But advisor's response overrides model's response.
Environment Spring AI version - 1.0.0-M2 Model - AzureOpenAIChatModel VectorDB - SimpleVectorStore
Steps to reproduce 1. Create a Chat response which has a advisor set 2. getUsage() in chatMetadata gives tokenCount as 0
Expected behavior It should provide the actual token count that model received
Bug might be in the class - DefaultChatClient.doGetChatResponse ChatResponse advisedResponse = chatResponse;
At this stage, advisedResponse has the right data. // apply the advisors on response if (!CollectionUtils.isEmpty(inputRequest.getAdvisors())) { var currentAdvisors = new ArrayList<>(inputRequest.getAdvisors()); for (RequestResponseAdvisor advisor : currentAdvisors) { advisedResponse = advisor.adviseResponse(advisedResponse, context); } } But after the for loop, advisedResponse is overwritten with the empty chatMetadata object
Thank You! -Chetan
Comment From: markpollack
We have significantly rewritten the Advisor infrastructure and have ran some tests and don't see this issue. Please reopen if 1.0 M3 doesn't solve this problem for you.
Comment From: chetan-galgali
Hey Mark - The issue is still not resolved. I tried with M3 and ChatResponseMetadata still has EmptyUsage object
Thanks Chetan