SpringBoot Cyclic dependency between @JsonComponents and ObjectMapper causes JsonComponents to be ignored

Spring Boot 2.7.5 + JUnit 5 Kotlin 1.7.20 openjdk version "17.0.4.1" 2022-08-12 LTS OpenJDK Runtime Environment Zulu17.36+17-CA (build 17.0.4.1+1-LTS) OpenJDK 64-Bit Server VM Zulu17.36+17-CA (build 17.0.4.1+1-LTS, mixed mode, sharing)

I need to use @Autowired in Jackson's JsonDeserializer (to deserialize event class, where its content is polymorphic and I need to create the correct instance using another Spring @Service bean). I discovered that you can use @JsonComponent to register (de)serializers as beans and @Autowired works within it.

So I have created two @JsonComponent annotated classes, one for the serializer and one for the deserializer. Only the deserializer has @Autowired in it.

Tests pass and everything works great.

Adding @Autowired also to the serializer class makes both serializer and deserializer classes to not work anymore, they are completely ignored. Json is serialized using Jackson's default serializer, not by my serializer. During deserialize, Jackson complains it can't find any JsonCreator method, even though it should be using my deserializer instead.

Removing all @Autowired statements from these two classes and creating completely new and different @JsonComponent serializer for totally different event class with @Autowired in it causes all the previous @JsonComponent (de)serializers to not work, including the first ones, which were working fine until this new one was introduced to the mix. (Which should indicate that it's not because the first two are for the same event class).

By trial and error, I found out that even having @Autowired in a only one of (de)serializers works. Having more than one in one class works. Having @Autowired in more than one (de)serializer class causes all of them to not work (I have println() in all (de)serialize methods and none of them shows up).

Code:

THIS WORKS:

open class MessagingEvent(val source: String, val timestamp: Long = System.currentTimeMillis())

class EventEntityCreated : MessagingEvent {
    val entityName: String
    val entity: Any

    constructor(source: String, timestamp: Long, entityName: String, entity: Any) : super(source, timestamp) {
        this.entity = entity
        this.entityName = entityName
    }

    override fun toString(): String {
        return "EventEntityCreated(entity=$entity, entityName='$entityName') ${super.toString()}"
    }
}

class EventEntityUpdated : MessagingEvent {
    val entityName: String
    val previousVersion: Resource
    val entity: Resource

    constructor(source: String, timestamp: Long, entityName: String, previousVersion: Resource, entity: Resource) : super(source, timestamp) {
        this.entityName = entityName
        this.previousVersion = previousVersion
        this.entity = entity
    }

    override fun toString(): String {
        return "EventEntityUpdated(previousVersion=$previousVersion, entity=$entity, entityName='$entityName') ${super.toString()}"
    }
}


@JsonComponent
internal class EventEntityCreatedDeserializer : JsonDeserializer<EventEntityCreated>() {
    init {
        println("EventEntityCreated deserializer init!")
    }

    // NOTE: this is uncommented
    @Autowired
    private lateinit var systemEntityConfiguration: SystemEntityConfiguration

    @Throws(IOException::class, JsonProcessingException::class)
    override fun deserialize(p: JsonParser, ctxt: DeserializationContext?): EventEntityCreated {
        println("Using custom deserializer for EventEntityCreated")

        val mapper: ObjectMapper = p.codec as ObjectMapper
        val node = mapper.readTree<JsonNode>(p)

        val source = node.get("source")?.let { if (it.isNull) null else it.textValue() } ?: ""
        val entityName = requireNotNull(
            node.get("entityName")?.let { if (it.isNull) null else it.textValue() }) { "entityName cannot be null" }
        val timestamp = node.get("timestamp").let { if (it.isNull) null else it.asLong() } ?: 0

        val systemEntityConfiguration = SystemEntityConfiguration(mapper) // I need to autowire this

        val entityType = systemEntityConfiguration.findEntity(entityName) ?: error("Entity type $entityName not found")

        val resClass: Class<*> = entityType.resourceClass()

        val entity = mapper.treeToValue(node.get("entity"), resClass) as Resource

        return EventEntityCreated(
            source, timestamp, entityName, entity
        )
    }
}

@JsonComponent
internal class EventEntityCreatedSerializer : JsonSerializer<EventEntityCreated>() {

// NOTE: if uncommented, everything breaks
//    @Autowired
//    private lateinit var systemEntityConfiguration: SystemEntityConfiguration

    @Throws(IOException::class, JsonProcessingException::class)
    override fun serialize(value: EventEntityCreated, gen: JsonGenerator, provider: SerializerProvider) {
        println("Using custom serializer for EventEntityCreated")

        val mapper: ObjectMapper = gen.codec as ObjectMapper
        val node = mapper.createObjectNode()
        node.put("source", value.source)
        node.put("timestamp", value.timestamp)
        node.put("entityName", value.entityName)
        node.set<JsonNode>("entity", mapper.valueToTree(value.entity))
        gen.writeTree(node)
    }
}

Uncommenting the @Autowired in EventEntityCreatedSerializer makes both of them to not work.

Now let's add the same classes for another entity.

@JsonComponent
internal class EventEntityUpdatedDeserializer : JsonDeserializer<EventEntityUpdated>() {
    init {
        println("EventEntityUpdated deserializer init!")
    }

// NOTE: if uncommented, everything breaks
//    @Autowired
//    private lateinit var systemEntityConfiguration: SystemEntityConfiguration

    @Throws(IOException::class, JsonProcessingException::class)
    override fun deserialize(p: JsonParser, ctxt: DeserializationContext?): EventEntityUpdated {
        println("Using custom deserializer for EventEntityUpdated")

        val mapper: ObjectMapper = p.codec as ObjectMapper
        val node = mapper.readTree<JsonNode>(p)

        val source = node.get("source")?.let { if (it.isNull) null else it.textValue() } ?: ""
        val entityName = requireNotNull(
            node.get("entityName")?.let { if (it.isNull) null else it.textValue() }) { "entityName cannot be null" }
        val timestamp = node.get("timestamp").let { if (it.isNull) null else it.asLong() } ?: 0

        val systemEntityConfiguration = SystemEntityConfiguration(mapper) // I need to autowire this

        val entityConfig = systemEntityConfiguration.findEntity(entityName) ?: error("Entity type $entityName not found")

        val resClass: Class<*> = entityConfig.resourceClass()

        val previousVersion = mapper.treeToValue(node.get("previousVersion"), resClass) as Resource
        val entity = mapper.treeToValue(node.get("entity"), resClass) as Resource

        return EventEntityUpdated(
            source, timestamp, entityName, previousVersion, entity
        )
    }
}

@JsonComponent
internal class EventEntityUpdatedSerializer : JsonSerializer<EventEntityUpdated>() {

// NOTE: if uncommented, everything breaks
//    @Autowired
//    private lateinit var systemEntityConfiguration: SystemEntityConfiguration

    @Throws(IOException::class, JsonProcessingException::class)
    override fun serialize(value: EventEntityUpdated, gen: JsonGenerator, provider: SerializerProvider) {
        println("Using custom serializer for EventEntityUpdated")

        val mapper: ObjectMapper = gen.codec as ObjectMapper
        val node = mapper.createObjectNode()
        node.put("source", value.source)
        node.put("timestamp", value.timestamp)
        node.put("entityName", value.entityName)
        node.set<JsonNode>("previousVersion", mapper.valueToTree(value.previousVersion))
        node.set<JsonNode>("entity", mapper.valueToTree(value.entity))
        gen.writeTree(node)
    }
}

If you copy all the code as-is, it works.

Uncommenting lateinit var ... in ONE of EventEntityUpdatedDeserializer, OR EventEntityUpdatedSerializer, OR EventEntityCreatedSerializer (so there are two @Autowireds in total) makes all of them to stop working.

What is causing this behavior? Is it a bug? Should @Autowired work in those classes in the first place?

P.S. I have no custom Jackson configuration in application.properties/yml, but I use this Customerizer:

@Configuration
open class JacksonConfiguration {
    @Bean
    open fun jsonCustomizer(): Jackson2ObjectMapperBuilderCustomizer? {
        logger.debug("Customizing ObjectMapper")
        return Jackson2ObjectMapperBuilderCustomizer { builder: Jackson2ObjectMapperBuilder ->
            builder
                .featuresToDisable(
                    SerializationFeature.INDENT_OUTPUT,
                    SerializationFeature.WRITE_DATES_AS_TIMESTAMPS,
                    SerializationFeature.FAIL_ON_EMPTY_BEANS,
                    MapperFeature.DEFAULT_VIEW_INCLUSION,
                )
                .featuresToEnable(
                    DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES,
                    MapperFeature.USE_STD_BEAN_NAMING,
                )
        }
    }
}

Commenting the Customizer bean doesn't change anything.

Comment From: daliborfilus

After hours and hours of fighting this thing I discovered that this breaks because my bean SystemEntityConfiguration requires ObjectMapper (to load huge json config file). Somehow it works if the SEC is autowired to just one (de)serializer, but breaks completely when I try to autowire it to more of them. Even with lateinit var. Removing ObjectMapper bean dependency from SystemEntityConfiguration resolves the issue.

But why it doesn't break "the usual way" with message like "Application context failed to load" and instead justs silently ignores the (de)serializers?

Comment From: wilkinsona

@daliborfilus I'm glad to hear that you figured this out. This sort of thing is one of the reasons why we strongly recommend constructor injection rather than field injection. An injected constructor argument should never be null whereas an @Autowired field can be left null in certain circumstances, for example when trying to break a dependency cycle or when a component is created extremely early and autowired support is not yet available.

Comment From: daliborfilus

That's the thing though. I used lateinit var in the code above because constructor injection didn't work. But it didn't work the same way, without any warning/error message.

When I figured out why it didn't work at all and got it to work (I instantiated my own ObjectMapper for the SystemEntityConfiguration dependency) I refactored the code to use constructor injection (I prefer it that way in the whole app).

When I try to add the cyclic dependency on ObjectMapper back, with constructor injection, it still just "doesn't work" without showing me any errors.

The JsonComponent instances are not registered anywhere, they are likely ignored, the test context boots without them and the test fails on "Cannot construct instance of cmdb.core.Resource (no Creators, like default constructor, exist):", like with the autowired lateinit var property. (The Resource is an interface and is used inside the entity class, as you can see in the first post. So it tries to deserialize the content using default jackson deserializer.)

It really doesn't say any error. Look:

2022-10-25 15:02:21.094  INFO 29009 --- [    Test worker] cmdb.core.event.EventEntityCreatedTest   : Starting EventEntityCreatedTest using Java 17.0.4.1 on Dalibors-MacBook-Pro-M1.local with PID 29009 (started by dfilus in /Users/dfilus/code/redacted/backend/lib-core-service)
2022-10-25 15:02:21.094 DEBUG 29009 --- [    Test worker] cmdb.core.event.EventEntityCreatedTest   : Running with Spring Boot v2.7.5, Spring v5.3.23
2022-10-25 15:02:21.095  INFO 29009 --- [    Test worker] cmdb.core.event.EventEntityCreatedTest   : The following 1 profile is active: "test"
2022-10-25 15:02:21.673 DEBUG 29009 --- [    Test worker] uration$$EnhancerBySpringCGLIB$$16a4f8fb : Customizing ObjectMapper
2022-10-25 15:02:21.853 DEBUG 29009 --- [    Test worker] c.core.system.SystemEntityConfiguration  : Loaded system-entities.json configuration file in 115 ms
2022-10-25 15:02:22.009  INFO 29009 --- [    Test worker] o.s.l.c.support.AbstractContextSource    : Property 'userDn' not set - anonymous context will be used for read-write operations
2022-10-25 15:02:22.049  INFO 29009 --- [    Test worker] cmdb.core.event.EventEntityCreatedTest   : Started EventEntityCreatedTest in 1.102 seconds (JVM running for 1.641)

Cannot construct instance of `cmdb.core.Resource` (no Creators, like default constructor, exist):

When I add println() to the deserialize method, it doesn't show up.

Comment From: wilkinsona

Thanks sounds like a bug. If you can provide a minimal sample that reproduces the problem , ideally written in Java so that we know that it isn't Kotlin-specific, we can take another look.

Comment From: daliborfilus

Here is the sample code: daliborfilus/spring-boot-issue-32863

Just autowiring ObjectMapper into the Deserializer class makes it being ignored without any error. In my code, it is a transitive dependency on ObjectMapper that causes the same behavior, but this is as minimal as I can get.

See the last commit - the tests work until you add the ObjectMapper inject in the Deserializer.

Comment From: wilkinsona

Thanks, @daliborfilus. I've reproduced the problem. The expected BeanCurrentlyInCreationException is thrown but, unfortunately, the bean factory suppresses it. That isn't desirable in this situation. We'll talk to the Framework team and see if there's anything that can be done to improve the situation.

Comment From: wilkinsona

The silent failure appears to be at least somewhat self-inflicted. The Module beans are manually retrieved at the moment rather than being injected. Things have been that since support for configuring the ObjectMapper with Module beans was first introduced but it's not clear why. Changing to dependency injection fixes the problem and results in the BeanCurrentlyInCreationException causing a failure. We'll make that change in 3.0.x. Unfortunately, we feel that it's too risky for 2.x.

Comment From: daliborfilus

Thanks!