On 30 May, Amazon announced the Converse API, which is intended to create an abstraction for accessing different models.
https://aws.amazon.com/about-aws/whats-new/2024/05/amazon-bedrock-new-converse-api/
For non-streaming responses, the Converse API can be used, while for streaming, the Converse Stream API can be used.
I would love to use this API with Spring AI.
Comment From: maxjiang153
Hi @Hakenadu could you show some scenarios to explain why you would use the Amazon Bedrock Converse API directly?
If you just use the converse you still need to pay attention to managing the parameters with different models, I'm thinking it might be best to keep current Spring-AI ways to call different models and change the native AWS BedrockRuntime SDK to use Bedrock Converse API
Comment From: Hakenadu
Hi @maxjiang153, we are currently implementing an API that allows access to different models from various providers (of which AWS Bedrock is just one), abstracting away the model choice for an end user.
By implementing the Converse API, I hope to have a more manageable set of beans (especially since we want to customise models backwards).
On the other hand, I can well imagine exposing the Converse API format, which is designed for abstraction, to the outside world. Specifically, I would try to reuse the model classes in our own API. In fact, I am currently doing this with the (Spring AI) OpenAI request models.
Of course, it may be that these are very specific framework conditions. Nevertheless, I believe that the Converse API can be a good addition to the Bedrock implementations in Spring AI.
Comment From: maxjiang153
Nice idea. I'm working on implementing a new BedrockConverseApi instead of the current AbstractBedrockApi, and replacing the current Bedrock ChatModelApi with the new BedrockConverseApi.
Comment From: Hakenadu
@maxjiang153 I already took a look into you changes, thank you for your work in this matter!!
I noticed that (also in the current implementation) it isn't possible to pass a custom BedrockRuntimeClient to the AbstractBedrockApi. Could you perhaps also directly include a corresponding constructor for this?
that would be great!
Comment From: maxjiang153
@Hakenadu Great idea, I'll add this feature to PR's todo list. Also if you have any ideas plz let me know.
Comment From: markpollack
This is an important issue.