Some LLM providers marketed as compatible with OpenAI API (e.g., Groq, OpenRouter), are in fact NOT fully compliant with the OpenAI API.
For example, sometime, they don't implement theuser
specification for the Chat Create Completion API and and as a result won't support user message contents like:
{
"role": "user",
"content": [
{
"type": "text",
"text": "You are a helpful assistant."
}
]
}
or like:
"role": "user",
"content": [
{
"type": "text",
"text": "What'\''s in this image?"
},
{
"type": "image_url",
"image_url": {
"url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
}
}
]
In other words those provides won't be able to serve and support multimodal models (as a reference Ollama does).
instead they support only the outdated (but still valid) shortcut text expression:
{
"role": "user",
"content": "You are a helpful assistant."
}
The goal of this PR is to support, such not fully compliant providers.
I'm also introducing a very simple integration test testing Groq and OpenRoter API basic compatibility.
This PR fixes #856 and #621.
Comment From: tzolov
@didalgolab , thank you for your contribution!
I've re-written the title and the description to better explain the context.
I also corrected the example because the discrepancy concerns only the user
not the system
messages?
The issue is that those providers don't implement the full OpenAI API partially. Though this is unfortunate I guess Spring AI should try to support them and your contribution is highly appreciated!
Will try to review it later today.
Comment From: tzolov
LGTM, thank you @didalgolab
Comment From: tzolov
rebased and merged at a6bed95358ab2cd5a3b3ec9a1a614a1b7ae610aa
Additional tests added with 51b1d719d51a667e11b8de2441875cbe280711fd