Affects: 6.1.3

Background

Many real-world REST API's have the following design: * If the request is successful, a 200 status is returned and the payload is a JSON document of one type * If the request fails, a 4xx status is returned and the payload is a JSON "error" document of a different type

This is very common - an example of this is Twilio's API. The examples below are from their documentation:

Request:

GET /2010-04-01/Accounts/ACXXXXX.../Messages/SM1f0e8ae6ade43cb3c0ce4525424e404f.json

Successful response (status 200):

{
    "sid": "SM1f0e8ae6ade43cb3c0ce4525424e404f",
    "date_created": "Fri, 13 Aug 2010 01:16:24 +0000",
    "date_updated": "Fri, 13 Aug 2010 01:16:24 +0000",
    "date_sent": null,
    "account_sid": "ACXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "to": "+15305431221",
    "from": "+15104564545",
    "body": "A Test Message",
    "status": "queued",
    "flags":["outbound"],
    "api_version": "2010-04-01",
    "price": null,
    "uri": "/2010-04-01/Accounts/ACXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/Messages/SM1f0e8ae6ade43cb3c0ce4525424e404f.json"
}

Error response (status 4xx):

{
  "status": 404,
  "message": "The requested resource was not found"
}

Here's the problem: When using Jackson's ObjectMapper, AFAICT there doesn't seem to be an easy way to tell Jackson to map the JSON to (for example) class Message.java in the first case, but class Error.java in the second case (or maybe I'm missing something?)

As a result, you have to just hard-wire Message.java, and if an error occurs then all of the error detail is lost because it can't be mapped by Jackson (or is mapped to nowhere).

Jackson does have some support for polymorphic/dynamic target Java type determination, but that support is limited and can't handle the above case.

But more crucially, the ObjectMapper has no notion of the HTTP status code context, which obviously is a key thing to know in this particular case - in fact, it's all you need to know!

Summary

In short, JSON documents don't come with as much implicit type information as other document types (e.g., XML, where the document tag is either <Message> or <RestException>). So when mapping JSON documents with Jackson, there is a clear need for more flexibility to make "runtime" determinatinon of the target document type.

Feature Request

Extend the functionality of MappingJackson2HttpMessageConverter to support dynamic (i.e., at HTTP response time) determination of the most correct target object type for the ObjectMapper via some callback interface.

Although in the above example this determination can be done simply by looking at the HTTP status response code (after casting message to a ClientHttpResponse), more generally this functionality should support arbitrary inspection of the JSON document if needed (e.g., to look for the presence of specific sub-fields, etc.).

That functionality could be implemented using a callback looking something like this:

    @FunctionalInterface
    public interface TypeMapper {

        /**
         * Determine the deserialization target type for the given input.
         *
         * @param message HTTP input message, from which the body has already been consumed
         * @param type the type that has been determined so far, before looking at the payload
         * @param node decoded JSON input
         * @return target type for Jackson deserialization
         * @throws IOException if an I/O error occurs
         * @throws MismatchedInputException if target type cannot be determined
         */
        Class<?> decodeType(HttpInputMessage message, JavaType type, JsonNode node) throws IOException, MismatchedInputException;
    }

Note that the above callback interface is simplified by the assumption that, if such a callback is defined, then MappingJackson2HttpMessageConverter is going to read the entire JSON document into memory (i.e., JsonNode) before invoking it. Unfortunately, this means document consumption is no longer an online process.

A better API design would let the implementer of the interface decide whether that's needed (perhaps there could be adefault method that reads into memory, but that implementors could override if they wanted to preserve online behavior).

In the Twilio API example, and probably most other real-world cases, reading into memory is probably not going to be needed because all you need to do is look at the HTTP status code.

Comment From: rstoyanchev

@archiecobbs it's a bit challenging to discuss a specific solution at the level of a component such as a message converter before considering what you might do at the client API level. What client are you using first of all?

Generally, the pattern is that you provide a target class to the client to indicate what object to decode to, and that has to be the Message.class expected for a success outcome. If there is an error, then error handling takes over, and you choose separate how to decode the error response. That's how we'd expect this to work, and depending on the client you use, there are ways to customize error handling.

Either way a client API cannot return one of two possible object types that don't have much in common, unless you return Object but in that case you have to check and downcast. That's less convenient than returning the object for the success case, and dealing with errors as exceptions.

Comment From: archiecobbs

What client are you using first of all?

I've been using RestTemplate.

However, I just discovered that the newer RestClient does support dynamic payload interpretation via HTTP status code via RestClient.Builder.defaultStatusHandler(), so that's a nice way to handle the normal vs. error distinction.

Generally, the pattern is that you provide a target class to the client to indicate what object to decode to, and that has to be the Message.class expected for a success outcome. If there is an error, then error handling takes over, and you choose separate how to decode the error response. That's how we'd expect this to work, and depending on the client you use, there are ways to customize error handling.

Either way a client API cannot return one of two possible object types that don't have much in common, unless you return Object but in that case you have to check and downcast. That's less convenient than returning the object for the success case, and dealing with errors as exceptions.

I agree downcasting from Object would be goofy. Instead, we want to take advantage of Java's type system. Put another way, there are situations where the thing you'd like to do is to map the returned JSON entity onto a non-trivial Java type hierarchy.

For example, Twilio's Message resource includes both SMS and MMS messages (as you probably know MMS is a newer form of SMS message that can have one or more media attachments, like photos). So every MMS message is an SMS message, but it also has additional capabilities. This is a natural fit a Java type with a sub-type.

The JSON looks almost the same - they are distinguished in JSON by whether their sid property starts with SM or MM, but in the case of MMS, there can also be media subresource URL's.

In Java, you'd want to model this resource like this:

public class SmsMessage {
    public String getDestPhoneNumber();
    public String getBody();
    ...
}
public class MmsMessage extends SmsMessage {
    public List<URI> getMediaUrls();
}

Then when querying, you'd invoke RestTemplate.getForObject(SmsMessage.class), and you'd want it to return either a SmsMessage or an MmsMessage object depending on the type of JSON resource returned.

But of course you can't (easily) do any of this today, because there's no way to instruct the converter when to target SmsMesasge vs. MmsMessage.

Comment From: rstoyanchev

I'm not sure how having access to the HttpInputMessage helps to decide which subclass to create. The response body can be consumed only once (unless it is buffered). so reading it to decide would get in the way of Jackson doing the same, plus you shouldn't have to parse JSON.

This is something that needs to be done with involvement from the JSON library. I think Jackson's polymorphic type handling feature is along those lines.

Comment From: archiecobbs

I'm not sure how having access to the HttpInputMessage helps to decide which subclass to create. The response body can be consumed only once (unless it is buffered). so reading it to decide would get in the way of Jackson doing the same, plus you shouldn't have to parse JSON.

As pointed out above, in some scenarios it's only possible to determine the appropriate target type by inspecting (some of) the JSON. So reading it into memory is required. In any case, it can be left up to the discriminator whether and how much it needs to do this. For example, it could buffer and rewind the input. The API would have to support doing that of course.

This is something that needs to be done with involvement from the JSON library. I think Jackson's polymorphic type handling feature is along those lines.

As also pointed out, Jackson's polymorphic type handling feature is insufficiently powerful to handle this. That's why this feature would be helpful for Spring to provide.

Comment From: rstoyanchev

Are you sure you've exhausted what can be done through Jackson's polymorphic type handling feature, and a custom TypeResolverBuilder for example? If it is indeed insufficient, perhaps consider how it can be enhanced or request such an enhancement. I see this as something more general than just HTTP and message converters.

In any case, I don't think we are going to work on such a feature, so I'm going to close this. HttpMessageConverter already provides access to the HttpInputMessage if you would like to do something more with that.

Comment From: archiecobbs

I see this as something more general than just HTTP and message converters.

On reflection I agree with this. This is a Jackson problem, not a Spring problem.

In any case, I've switch to Gson :)

Comment From: rstoyanchev

@archiecobbs, it may be of interest to others to share if Gson has the polymorphic feature you need.

Comment From: archiecobbs

it may be of interest to others to share if Gson has the polymorphic feature you need

Both solutions support polymorphism. Jackson supports a few common such use cases through annotations, but in the general case both solutions require writing code implementing some "TypeHandler" interface of some sort that processes a token stream or whatever.

Having said that, (getting into personal opinion here) Gson's API is about 3% of the size of Jackson's and much simpler & cleaner to deal with. Jackson's API is too sprawling for a relatively simple use case like mine.