This pull request introduces a new module to the Spring AI framework, designed to integrate and expand the use of Anthropic Claude LLM models within the VertexAI platform. The integration leverages the Model Garden feature of VertexAI to provide seamless and consistent access to Claude LLMs.
Comment From: alessiobertazzo88
Hi @tzolov @markpollack just to understand if you could evaluate the merge of this PR that enables the use Anthropic's models from GCP Vertex Model Garden that has variations from both Anthropic standard endpoints but also from Vertex already implemented (Gemini) modules.
Thanks in advance!
Comment From: markpollack
Hi, sorry for letting this sit so long. This is a pretty big PR and i'm growing concerned over the number of models and variations each with different code bases. Just so verify my understanding with you, the need for this new API as compared to using the Vertex API to access the model is that the Anthropic model you want to use is different than the one exposed via Vertex API?
Comment From: alessiobertazzo88
Hi, that's correct. Initially, I tried using the Vertex API to take advantage of Google’s SDK and also benefit from the authentication component. I had also seen the possibility of changing the publisher by dynamically specifying the model in the URL. Unfortunately, it’s not possible to use custom models that the Anthropic APIs within VertexAI’s model garden require (here’s the link with the details: https://console.cloud.google.com/vertex-ai/publishers/anthropic/model-garden/claude-3-5-sonnet?_ga=2.151856633.1780909182.1726601103-828101531.1719608013). This is the reason why I created this additional version.
Comment From: ddobrin
Hi @alessiobertazzo88 could you please confirm the following, just to be sure that I understand this right?
You pick Sonnet 3.5 for example from the Model Garden in Vertex AI, Enable the API and would like to access it programmatically from Spring AI and a supporting SDK.
Comment From: alessiobertazzo88
Hi @ddobrin Correct! The Google SDK now is only used for authentication because of lack of models for Anthropic use.
Comment From: ddobrin
@alessiobertazzo88 thanks for the confirmation.
I am waiting for a small change to be made in Spring AI for another issue, which should help validate this