Bug description All calls to OpenAIs chat completion since some hours seem to fail, since the reply from openai now includes the refusal field. In normal situations this field is returned "null", which is a bug from openai or a new feature, i guess? So, for me, nothing is working anymore.
Environment Spring AI version 1.0.0-SNAPSHOT Java version 17 OpenAI tested with function calls and gpt-4o and -mini
Steps to reproduce I assume, just test against current chat completion api
Expected behavior field is recognized
Comment From: mase-ppi
Exception looks like:
org.springframework.web.client.RestClientException: Error while extracting response for type [org.springframework.ai.openai.api.OpenAiApi$ChatCompletion] and content type [application/json] at org.springframework.web.client.DefaultRestClient.readWithMessageConverters(DefaultRestClient.java:236) ~[spring-web-6.1.5.jar:6.1.5] at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.readBody(DefaultRestClient.java:667) ~[spring-web-6.1.5.jar:6.1.5] at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.toEntityInternal(DefaultRestClient.java:637) ~[spring-web-6.1.5.jar:6.1.5] at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.toEntity(DefaultRestClient.java:626) ~[spring-web-6.1.5.jar:6.1.5] at org.springframework.ai.openai.api.OpenAiApi.chatCompletionEntity(OpenAiApi.java:920) ~[spring-ai-openai-1.0.0-20240805.173614-443.jar:1.0.0-SNAPSHOT] at org.springframework.ai.openai.OpenAiChatModel.lambda$call$1(OpenAiChatModel.java:224) ~[spring-ai-openai-1.0.0-20240805.173614-443.jar:1.0.0-SNAPSHOT] at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:335) ~[spring-retry-2.0.5.jar:na] at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:211) ~[spring-retry-2.0.5.jar:na] at org.springframework.ai.openai.OpenAiChatModel.lambda$call$3(OpenAiChatModel.java:224) ~[spring-ai-openai-1.0.0-20240805.173614-443.jar:1.0.0-SNAPSHOT] at io.micrometer.observation.Observation.observe(Observation.java:565) ~[micrometer-observation-1.12.4.jar:1.12.4] at org.springframework.ai.openai.OpenAiChatModel.call(OpenAiChatModel.java:221) ~[spring-ai-openai-1.0.0-20240805.173614-443.jar:1.0.0-SNAPSHOT] ..... at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na] at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na] at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:255) ~[spring-web-6.1.5.jar:6.1.5] at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:188) ~[spring-web-6.1.5.jar:6.1.5] at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118) ~[spring-webmvc-6.1.5.jar:6.1.5] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:925) ~[spring-webmvc-6.1.5.jar:6.1.5] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:830) ~[spring-webmvc-6.1.5.jar:6.1.5] at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-6.1.5.jar:6.1.5] at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1089) ~[spring-webmvc-6.1.5.jar:6.1.5] at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:979) ~[spring-webmvc-6.1.5.jar:6.1.5] at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014) ~[spring-webmvc-6.1.5.jar:6.1.5] at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:914) ~[spring-webmvc-6.1.5.jar:6.1.5] at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:590) ~[tomcat-embed-core-10.1.19.jar:6.0] at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885) ~[spring-webmvc-6.1.5.jar:6.1.5] at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658) ~[tomcat-embed-core-10.1.19.jar:6.0] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:205) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) ~[tomcat-embed-websocket-10.1.19.jar:10.1.19] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-6.1.5.jar:6.1.5] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.5.jar:6.1.5] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-6.1.5.jar:6.1.5] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.5.jar:6.1.5] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-6.1.5.jar:6.1.5] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.5.jar:6.1.5] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:391) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1744) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) ~[tomcat-embed-core-10.1.19.jar:10.1.19] at java.base/java.lang.Thread.run(Thread.java:833) ~[na:na] Caused by: org.springframework.http.converter.HttpMessageNotReadableException: JSON parse error: Unrecognized field "refusal" (class org.springframework.ai.openai.api.OpenAiApi$ChatCompletionMessage), not marked as ignorable at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:406) ~[spring-web-6.1.5.jar:6.1.5] at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.read(AbstractJackson2HttpMessageConverter.java:354) ~[spring-web-6.1.5.jar:6.1.5] at org.springframework.web.client.DefaultRestClient.readWithMessageConverters(DefaultRestClient.java:213) ~[spring-web-6.1.5.jar:6.1.5] ... 64 common frames omitted Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "refusal" (class org.springframework.ai.openai.api.OpenAiApi$ChatCompletionMessage), not marked as ignorable (5 known properties: "content", "name", "tool_call_id", "role", "tool_calls"]) at [Source: (org.springframework.util.StreamUtils$NonClosingInputStream); line: 13, column: 8] (through reference chain: org.springframework.ai.openai.api.OpenAiApi$ChatCompletion["choices"]->java.util.ArrayList[0]->org.springframework.ai.openai.api.OpenAiApi$ChatCompletion$Choice["message"]->org.springframework.ai.openai.api.OpenAiApi$ChatCompletionMessage["refusal"]) at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:1138) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:2224) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1719) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperties(BeanDeserializerBase.java:1669) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:546) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1419) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:570) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:439) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1419) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:359) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:570) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:439) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1419) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2105) ~[jackson-databind-2.15.4.jar:2.15.4] at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1481) ~[jackson-databind-2.15.4.jar:2.15.4] at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:395) ~[spring-web-6.1.5.jar:6.1.5] ... 66 common frames omitted
Comment From: TarasVovk669
I have the same issue
Comment From: cocomongg
me too. I have the same issue.
Comment From: tzolov
@cocomongg , @TarasVovk669 I'm unable to reproduce this issue nor our OpenAI integration tests.
The OpenAI API reference documentation does not mention "refusal" field either.
So i'm reluctant to introduce new non documented field.
Would the @JsonIgnoreProperties(ignoreUnknown = true) annotation on the ChatCompletionMessage record resolve the error for you?
Comment From: tzolov
Can you please add the spring.jackson.deserialization.fail_on_unknown_properties=false property to your application.properties file and let me know if it resolves the problem?
Comment From: mase-ppi
@tzolov did you test against an own gpt-deployment or official openai hosted? It also needs to be 4o or 4o-mini.
i think it is something like: https://platform.openai.com/docs/guides/structured-outputs/refusals
but in our case, refusal ist just delivered as null.
But yes, would make sense to ignore unknown fields in order to be more rebust against new changes.
Thanks
Comment From: tzolov
@mathekcbo, thank you for the reference. I'll have a look
Comment From: mase-ppi
@tzolov the property didnt work for me - just to clarify, I tried:
spring:
jackson:
deserialization:
fail_on_unknown_properties: false
Comment From: tzolov
@mathekcbo , thank you for testing.
Regarding:
@tzolov did you test against an own gpt-deployment or official openai hosted? It also needs to be 4o or 4o-mini.
Our ITs are running against the official OpenAi hosted and they include 4o and 4o-mini.
I'm puzzled why I'm unable to reproduce it. What user question would cause the "refusal" field to be set?
Comment From: mase-ppi
@mathekcbo , thank you for testing.
Regarding:
@tzolov did you test against an own gpt-deployment or official openai hosted? It also needs to be 4o or 4o-mini.
Our ITs are running against the official OpenAi hosted and they include 4o and 4o-mini.
I'm puzzled why I'm unable to reproduce it. What user question would cause the "refusal" field to be set?
"How are you" (in my case: "Wie gehts dir?") - acutally i cannot see any prompt not returning it so far
Comment From: mase-ppi
@tzolov any possibility to print out my stuff going to the openai api without debugging everything? Maybe the request/options are different, as only a few users seem to have this problem
Comment From: TarasVovk669
I added an annotation to the model
Comment From: tzolov
related to https://github.com/spring-projects/spring-ai/issues/1196
Comment From: tzolov
I've added PR to support the new openai structured outputs features: https://github.com/spring-projects/spring-ai/pull/1198
For the new "refusal" response message field I guess @TarasVovk669 's PR https://github.com/spring-projects/spring-ai/pull/1180 would cover the low level support.
Still I'm puzzled how to map this "refusal" field to the SpringAI AsistantMessage. It it seems that when the refusal is set the assistant message content is empty.
Comment From: tzolov
For now i will add it to the AssistantMessage metadata map.