Affects: 5.1.12

With #24009 fixed and versions updated, there's still issues with decoding Smile-encoded stream. Looks like a missyncronization between use of generator and parser:

org.springframework.core.codec.DecodingException: JSON decoding error: Invalid shared name reference 6; 
only got 2 names in buffer (invalid content); nested exception is
com.fasterxml.jackson.core.JsonParseException: Invalid shared name reference 6; only got 2 names in buffer (invalid content)
 at [Source: UNKNOWN; line: -1, column: 6843033]
    at org.springframework.http.codec.json.Jackson2Tokenizer.tokenize(Jackson2Tokenizer.java:99)
    at reactor.core.publisher.FluxMapSignal$FluxMapSignalSubscriber.onNext(FluxMapSignal.java:137)

Comment From: feutche

Test project attached: test-spring-smile.zip

Comment From: poutsma

I'm not sure it's a synchronisation issue. I disabled the USE_THREAD_LOCAL_FOR_BUFFER_RECYCLING feature for Jackson, which has been causing sync issues in the past, but it made no difference.

Maybe it was the "shared name reference" mentioned in the exception that put you on the track of synchronisation, but that is a feature of Smile.

Moreover, I was able to reproduce this issue without using any Spring code, see https://github.com/poutsma/jackson-smile-issue. It does use Reactor, but I suspect the issue is in Jackson itself.

@feutche As the original reporter, could you please report this Jackson issue here, possibly pointing to my repo?

Comment From: poutsma

Closed as invalid, as the issue can be reproduced without Spring code.

Comment From: feutche

Thanks for looking into it. Sorry I wasn't clear as I didn't mean multithreading/concurrency issue but more of maybe inconsistent use of generator and parser so their shared name buffers aren't in sync.

I still think it's the case - in your example (in your repo) - ObjectMapper.writeValuesAsBytes creates new SmileGenerator for every object (!) however there's a single Smile parser created for the whole flow (which is the way it should be, I guess). So obviously their _seenNames buffers are not in sync. In this case the problem manifests itself when parser buffer overflows and being reset (with expectation that generator did the same, but it didn't, every new generator will only have 2 values in there, name0 and name1). However I've seen cases when backreference is just invalid and Json parser fails trying to assign value to a wrong field (that one drove me mad :), now at least I finally understand where it comes from!).

Comment From: feutche

So I think this issue should be reopened, the problem is not in Jackson but in the way it's inconsistently used between encoder and decoder.

Comment From: poutsma

Ok, reopening, but I won't have time to look into this until after my Christmas break.

Comment From: poutsma

Thanks for spotting this. We are now using a Jackson SequenceWriter when streaming, which fixes this issue.