Hi Spring team. Via WebMvcConfigurer#configureMessageConverters I included StringHttpMessageConverter (for reference: to avoid Springdoc OAS3 responses to get treated by Jackson).

Now, in my case I have a ResponseBodyAdvice defined which alters the return type of all operations by wrapping responses. This breaks my application as it seems that AbstractMessageConverterMethodProcessor only looks at the type of the actual target operation to decide which HttpMessageConverter supports said return type.

If the original operation response is of type object, nothing breaks as an object is simply wrapped by the ResponseBodyAdvice in another object (type remains object). But if the response is a string, the string is wrapped in an object by the ResponseBodyAdvice so the actual target type changes from object to string. However, the StringHttpMessageConverter still remains selected as a supported converter because this selection was based on the result type of the operation (string, namely) and does not look at the response type of the ResponseBodyAdvice. This ultimately results in a ClassCastException as AbstractMessageConverterMethodProcessor feeds the Object to StringHttpMessageConverter.

To me the expected behavior would be that AbstractMessageConverterMethodProcessor takes the (potentially) altered return type of a ResponseBodyAdvice into account to decide which HttpMessageConverters are supported. Do you agree?

Comment From: rstoyanchev

This part of the contract is not clearly spelled out but ResponseBodyAdvice wasn't meant to influence the decision about which converter to use. Keep in mind that converters take into account both the actual value as well as the type information from the controller method signature which can contain generic types. In that sense the MethodParameter and the actual value are expected to match each other.

I don't fully understand what you are trying to achieve but you cannot rely on ResponseBodyAdvice in this way.

Comment From: koen-serneels

To summarize what I'm doing: the application captures the responses of all controllers in a ResponseBodyAdvice and wraps these responses in another composite response (header/body) to allow a uniform response structure. Our controllers do not return JSON but always a real Object (DTO). So the ResponseBodyAdvice gets a response of type DTO-X (the actual data) and returns another DTO-Y which is a composite of some extra stuff (call it the header data) and the original DTO-X (the actual data). This composite is then transformed to JSON viaMappingJackson2HttpMessageConverter.

This works fine until SpringDoc OpenApiResource came along which returns JSON instead of the actual Object or ReponseEntity. Because of this the JSON it returns is now escaped by MappingJackson2HttpMessageConverter and is no longer a valid OpenAPI document.

So, one option would be to add StringHttpMessageConverter prior in the list to MappingJackson2HttpMessageConverter. BUT we also have controllers ourselves that return responses of type String, this String is still wrapped in the composite DTO-Y and still requires MappingJackson2HttpMessageConverter but now it's treated by StringHttpMessageConverter instead as the converter mechanism looks at the response type of the controller method signature.

But fair enough I understand that changing the response types of the controller methods via a ResponseBodyAdvice is not how it was intended and this is merely a side effect of off spec usage. So I'm closing the issue.

So the only solution to this would be to 'enforce' that all controllers actually have a return type of DTO-Y in there signature I suppose?

In the mean time I worked around the issue by introducing a delegating HttpMessageConverter which decides if this is a SpringDoc response needed to be processed by StringHttpMessageConverter or a non-SpringDoc response which is processed by MappingJackson2HttpMessageConverter.

Comment From: rstoyanchev

Okay I think I better understand now. By default StringHttpMessageConverter is registered ahead of MappingJackson2HttpMessageConverter and it renders String as is with any media type. For application/json that means by default we favor the case where the String represents serialized JSON rather than a JSON String. FWIW I think even if a String is valid JSON it makes sense for controllers to return an Object container of some sort to avoid this ambiguity.