Bug description Bedrock/Titan chats return token usage information which is deserialized into a TitanChatResponse record. Token usage information is dropped in the BedrockTitanChatModel class when converting TitanChatResponse objects to ChatResponse objects.

    public ChatResponse call(Prompt prompt) {
        TitanChatBedrockApi.TitanChatResponse response = this.chatApi.chatCompletion(this.createRequest(prompt));
        List<Generation> generations = response.results().stream().map((result) -> {
            return new Generation(result.outputText());
        }).toList();
        return new ChatResponse(generations);
    }

Environment All versions of Spring AI supporting Bedrock/Titan. Currently testing with M3-PLATFORM.

Steps to reproduce Execute any valid Chat against a Chat based Bedrock/Titan model using a BedrockTitanChatModel and validate that usage meta data is empty.

Eg:

BedrockTitanChatModel = new BedrockTitanChatModel(...);
ChatClient.builder(chatModel).defaultOptions(options).build();
ChatResponse response = client.prompt(prompt).call().chatResponse();
Usage usage = response.getMetadata().getUsage();
assertThat(usage.getPromptTokens()).isEqualTo(0);

Expected behavior Expecting that the usage in will be non-zero.

assertThat(usage.getPromptTokens()).isGreaterThan(0);

Comment From: markpollack

  1. thanks for reporting this. Note, we now support the bedrock converse api that also allows access to titan. We should check that this issue is not repeated in that code pathway.