Can Spring AI support the Qwen large language model? And can spring-ai-ollama-spring-boot-starter support function calling?

Comment From: qinfengge

For the first one, it supports Qwen. For the second one, it does not yet support ollama's function call.

Comment From: zzllkk2003

For the first one, it supports Qwen. For the second one, it does not yet support ollama's function call.

Which Maven package can I use?

Comment From: codespearhead

  1. Does Spring AI support the Qwen large language model?

Spring AI supports Ollama, so you just have to use the Ollama starter package.

  1. Does spring-ai-ollama-spring-boot-starter support function calling?

Spring AI does [1][2][3], but Ollama doesn't officially support function calling yet [4][5], so naturally spring-ai-ollama-spring-boot-starter doesn't have it baked-in. By the way, neither vLLM nor SGLang do either [6].

You might want to take a look at LangChain4j [8][9].

[1] https://github.com/spring-projects/spring-ai/discussions/193 [2] https://www.devturtleblog.com/sring-ai-function-calling-tutorial/ [3] https://spring.io/blog/2024/03/06/function-calling-in-java-and-spring-ai-using-the-latest-mistral-ai-api

[4] https://github.com/QwenLM/Qwen2/issues/378 (https://github.com/QwenLM/Qwen2/issues/378#issuecomment-2097355347) [5] https://github.com/ollama/ollama/issues/3137

[6] https://github.com/QwenLM/Qwen2/blob/d439e04e84ebe3734dea4e9769e17f380a8c72b8/README.md?plain=1#L192 [7] https://docs.langchain4j.dev/tutorials/tools#:~:text=following%20models%20have%20tool%20support

[8] https://docs.langchain4j.dev/tutorials/tools/#:~:text=following%20models%20have%20tool%20support [9] https://docs.langchain4j.dev/tutorials/spring-boot-integration/#langchain4j-spring-boot-starter

Comment From: jidaojiuyou

  1. spring cloud alibaba ai for qwen
  2. 目前无法支持。因为 ollama 的 api 不支持 tools 和 tools_choice 字段。但是可以参考以下文档自己实现。（不推荐但是可用，以下两种方式都可以实现ollama的function call。） langchain ollama function call qwen agent funciton call

Comment From: tzolov

@zzllkk2003, Ollama added (partial) function calling support last week and Spring AI just provided support for it: https://docs.spring.io/spring-ai/reference/api/chat/ollama-chat.html#_function_calling

But of it to work the underling models must support function calling. I'm not sure qwen2 supports it? Can you point me to a model that does? Just tried with qwen2 and got: 

{"error":"qwen2 does not support tools"}

For this to work we need a qwen/qwen2 model that has been trained for function calling?

Comment From: SpiritCloude

Llama 3.1-8B、70B、405B is a new state-of-the-art model and support Tools.but Spring AI does support function calling ，It would be great if the next step could support function calling

Comment From: zzllkk2003

For this to work we need a qwen/qwen2 model that has been trained for function calling?

@tzolov tzolov please see at here https://qwen.readthedocs.io/en/latest/framework/function_call.html

Comment From: markpollack

@SpiritCloude you can call the llama models with spring ai and function calling.

Comment From: salmar

I've tried function calling with the following models: - Qwen2 - 7B - Qwen 2.5 - 7B - Llama 3.1 - 8B - Llama 3.2 - 3B - Llama 3.3 - 70B

They all work as expected.

Comment From: markpollack

Thanks @salmar !!! closing the issue.