Bug description Error message:
{
"error": {
"message": "Invalid value: '{\"t...\"}}'. Supported values are: 'none', 'auto', and 'required'.",
"type": "invalid_request_error",
"param": "tool_choice",
"code": "invalid_value"
}
}
The toolChoice of OpenAiChatOptions is defined as a String, but in fact, it should be an Object. It can be a String or a Map. As defined in ToolChoiceBuilder, but now I can't set it to map. Open ai explains the toolChoice field as follows:
Controls which (if any) tool is called by the model. none means the model will not call any tool and instead generates a message. auto means the model can pick between generating a message or calling one or more tools. required means the model must call one or more tools. Specifying a particular tool via {"type": "function", "function": {"name": "my_function"}} forces the model to call that tool.
Reference link: https://platform.openai.com/docs/api-reference/chat/create
Environment JDK: 21 Spring AI: 1.0.0-M4 OS: macOS 15.0.1
Steps to reproduce
@SpringBootTest
public class ToolChoiceTest {
@Resource
private OpenAiChatModel openAiChatModel;
@Resource
private ObjectMapper objectMapper;
@Test
void test() throws JsonProcessingException {
Object currentWeather = OpenAiApi.ChatCompletionRequest.ToolChoiceBuilder.FUNCTION("CurrentWeather");
OpenAiChatOptions chatOptions = OpenAiChatOptions.builder()
.withFunctionCallbacks(
List.of(FunctionCallback.builder()
.description("Get the weather in location")
.function("CurrentWeather", new MockWeatherService())
.inputType(MockWeatherService.Request.class)
.build()))
// .withToolChoice(OpenAiApi.ChatCompletionRequest.ToolChoiceBuilder.AUTO) // ok
// .withToolChoice(OpenAiApi.ChatCompletionRequest.ToolChoiceBuilder.NONE) // ok
// .withToolChoice(OpenAiApi.ChatCompletionRequest.ToolChoiceBuilder.FUNCTION("CurrentWeather")) // compilation failed
.withToolChoice(objectMapper.writeValueAsString(currentWeather)) // NonTransientAiException: 400
// request body: {"messages":[{"content":"What is the temperature in Shanghai now?","role":"user"}],"model":"gpt-4o","stream":false,"temperature":0.7,"tools":[{"type":"function","function":{"description":"Get the weather in location","name":"CurrentWeather","parameters":{"$schema":"https://json-schema.org/draft/2020-12/schema","type":"object","properties":{"location":{"type":"string"},"unit":{"type":"string","enum":["C","F"]}}}}}],"tool_choice":"{\"type\":\"function\",\"function\":{\"name\":\"CurrentWeather\"}}"}
// response body: {"error":{"message":"Invalid value: '{\"t...\"}}'. Supported values are: 'none', 'auto', and 'required'.","type":"invalid_request_error","param":"tool_choice","code":"invalid_value"}}
.build();
chatOptions.setProxyToolCalls(true);
ChatResponse chatResponse = openAiChatModel.call(new Prompt("What is the temperature in Shanghai now?", chatOptions));
System.out.println(chatResponse.getResult().getOutput());
}
public static class MockWeatherService implements Function<MockWeatherService.Request, MockWeatherService.Response> {
public enum Unit { C, F }
public record Request(String location, Unit unit) {}
public record Response(double temp, Unit unit) {}
public Response apply(Request request) {
return new Response(30.0, Unit.C);
}
}
}
From the request body, it can be seen that tool_choice is serialized:
"tool_choice":"{\"type\":\"function\",\"function\":{\"name\":\"CurrentWeather\"}
open ai response:
{
"error": {
"message": "Invalid value: '{\"t...\"}}'. Supported values are: 'none', 'auto', and 'required'.",
"type": "invalid_request_error",
"param": "tool_choice",
"code": "invalid_value"
}
}
Expected behavior Change the toolChoice of OpenAiChatOptions to Object type
After the change After I changed the toolChoice of OpenAiChatOptions to type Object, the following code worked:
@Test
void test() throws JsonProcessingException {
OpenAiChatOptions chatOptions = OpenAiChatOptions.builder()
.withFunctionCallbacks(
List.of(FunctionCallback.builder()
.description("Get the weather in location")
.function("CurrentWeather", new MockWeatherService())
.inputType(MockWeatherService.Request.class)
.build()))
.withToolChoice(OpenAiApi.ChatCompletionRequest.ToolChoiceBuilder.FUNCTION("CurrentWeather")) // ok
// request body: {"messages":[{"content":"What is the temperature in Shanghai now?","role":"user"}],"model":"gpt-4o","stream":false,"temperature":0.7,"tools":[{"type":"function","function":{"description":"Get the weather in location","name":"CurrentWeather","parameters":{"$schema":"https://json-schema.org/draft/2020-12/schema","type":"object","properties":{"location":{"type":"string"},"unit":{"type":"string","enum":["C","F"]}}}}}],"tool_choice":{"function":{"name":"CurrentWeather"},"type":"function"}}
// response body: {"id":"chatcmpl-AcvLRP5ikO8N2ln9DVY8ZbBuhHwHw","object":"chat.completion","created":1733840261,"model":"gpt-4o-2024-08-06","choices":[{"index":0,"message":{"role":"assistant","content":null,"tool_calls":[{"id":"call_VPXdnJqOEbUWmrk1WEGfTPlE","type":"function","function":{"name":"CurrentWeather","arguments":"{\"location\":\"Shanghai\"}"}}],"refusal":null},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":70,"completion_tokens":5,"total_tokens":75,"prompt_tokens_details":{"cached_tokens":0,"audio_tokens":0},"completion_tokens_details":{"reasoning_tokens":0,"audio_tokens":0,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"system_fingerprint":"fp_9d50cd990b"}
.build();
chatOptions.setProxyToolCalls(true);
ChatResponse chatResponse = openAiChatModel.call(new Prompt("What is the temperature in Shanghai now?", chatOptions));
System.out.println(chatResponse.getResult().getOutput());
}
After changing toolChoice to type Object, I can use ToolChoiceBuilder directly
OpenAiApi.ChatCompletionRequest.ToolChoiceBuilder.FUNCTION("CurrentWeather")
In the final request body, toolChoice is:
"tool_choice":{"function":{"name":"CurrentWeather"},"type":"function"}
Comment From: zmwei666