Spring AI Unable to get promptTokens, completionTokens, and totalTokens in Stream mode

Getting promptTokens, completionTokens and totalTokens from Usage has always been 0 in Stream mode. How can I get tokens when Streaming?

If I use below code: String systemMsg = "Please answer user's question based on the knowledge, and today is {current_date}.";

    this.chatClient = ChatClient.builder(chatModel)
            .defaultOptions(Objects.requireNonNull(prompt.getOptions()))
            .defaultSystem(systemMsg)
            .defaultAdvisors(
                    new MessageChatMemoryAdvisor(chatMemory, conversationId, 10)
            )
            .build();

    String promptMsg = """
            This is the knowledge_base information:
            ---------------------
            {question_answer_context}
            ---------------------
    """;

    return this.chatClient.prompt()
            .system(s -> s.params(Map.of("current_date", LocalDate.now().toString())))
            .user(u -> u.text(prompt.getContents()))
            .advisors(
                    new QuestionAnswerAdvisor(vectorStore, searchRequest, promptMsg)
            )
            **.stream()**
            .chatResponse();

Part of ChatResponse is blow: {"metadata" : { "model" : "gpt-4o-2024-08-06", "rateLimit" : { "requestsLimit" : 0, "requestsRemaining" : 0, "requestsReset" : 0.0, "tokensRemaining" : 0, "tokensLimit" : 0, "tokensReset" : 0.0 }, "usage" : { "promptTokens" : 0, "completionTokens" : 0, "totalTokens" : 0, "nativeUsage" : { "promptTokens" : 0, "totalTokens" : 0, "completionTokens" : 0 }, "generationTokens" : 0 }, "promptMetadata" : [ ], "empty" : true }}

**But when I use no stream ** String systemMsg = "Please answer user's question based on the knowledge, and today is {current_date}.";

    this.chatClient = ChatClient.builder(chatModel)
            .defaultOptions(Objects.requireNonNull(prompt.getOptions()))
            .defaultSystem(systemMsg)
            .defaultAdvisors(
                    new MessageChatMemoryAdvisor(chatMemory, conversationId, 10)
            )
            .build();

    String promptMsg = """
            This is the knowledge_base information:
            ---------------------
            {question_answer_context}
            ---------------------
    """;

    return this.chatClient.prompt()
            .system(s -> s.params(Map.of("current_date", LocalDate.now().toString())))
            .user(u -> u.text(prompt.getContents()))
            .advisors(
                    new QuestionAnswerAdvisor(vectorStore, searchRequest, promptMsg)
            )
            **.call()**
            .chatResponse();

Then I can get the tokens {"metadata" : { "model" : "gpt-4o-2024-08-06", "rateLimit" : { "requestsLimit" : 0, "requestsRemaining" : 0, "requestsReset" : 0.0, "tokensRemaining" : 0, "tokensLimit" : 0, "tokensReset" : 0.0 }, "usage" : { "promptTokens" : 1649, "completionTokens" : 261, "totalTokens" : 1910, "nativeUsage" : { "promptTokens" : 0, "totalTokens" : 0, "completionTokens" : 0 }, "generationTokens" : 0 }, "promptMetadata" : [ ], "empty" : true }}

How can I fix this issue?

Comment From: FredLiu0813

I found the answer in previous issues, please close this issue, thanks. when streaming, set streamUsage=true in options.