Since updating our config server to Spring Boot 2.6.4 and Spring Cloud 2021.0.1 we get the following error when trying to retrieve a large (500KB) config file via GET http://host:8888/config-file/dev.
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 1024
at org.yaml.snakeyaml.reader.StreamReader.peek(StreamReader.java:136)
at org.yaml.snakeyaml.scanner.ScannerImpl.scanToNextToken(ScannerImpl.java:1222)
at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:308)
at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:248)
at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingKey.produce(ParserImpl.java:634)
at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:165)
at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:59)
at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:45)
at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:140)
at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:119)
at org.yaml.snakeyaml.composer.Composer.composeScalarNode(Composer.java:214)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:184)
at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:314)
at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:305)
at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:286)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:188)
at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:314)
at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:305)
at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:286)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:188)
at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:314)
at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:305)
at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:286)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:188)
at org.yaml.snakeyaml.composer.Composer.getNode(Composer.java:115)
at org.yaml.snakeyaml.constructor.BaseConstructor.getData(BaseConstructor.java:135)
at org.springframework.boot.env.OriginTrackedYamlLoader$OriginTrackingConstructor.getData(OriginTrackedYamlLoader.java:99)
at org.yaml.snakeyaml.Yaml$1.next(Yaml.java:514)
at org.springframework.beans.factory.config.YamlProcessor.process(YamlProcessor.java:198)
at org.springframework.beans.factory.config.YamlProcessor.process(YamlProcessor.java:166)
at org.springframework.boot.env.OriginTrackedYamlLoader.load(OriginTrackedYamlLoader.java:84)
at org.springframework.boot.env.YamlPropertySourceLoader.load(YamlPropertySourceLoader.java:50)
at org.springframework.boot.context.config.StandardConfigDataLoader.load(StandardConfigDataLoader.java:54)
at org.springframework.boot.context.config.StandardConfigDataLoader.load(StandardConfigDataLoader.java:36)
at org.springframework.boot.context.config.ConfigDataLoaders.load(ConfigDataLoaders.java:107)
at org.springframework.boot.context.config.ConfigDataImporter.resolveAndLoad(ConfigDataImporter.java:86)
at org.springframework.boot.context.config.ConfigDataEnvironmentContributors.withProcessedImports(ConfigDataEnvironmentContributors.java:116)
at org.springframework.boot.context.config.ConfigDataEnvironment.processInitial(ConfigDataEnvironment.java:240)
at org.springframework.boot.context.config.ConfigDataEnvironment.processAndApply(ConfigDataEnvironment.java:227)
at org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor.postProcessEnvironment(ConfigDataEnvironmentPostProcessor.java:102)
at org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor.applyTo(ConfigDataEnvironmentPostProcessor.java:201)
at org.springframework.cloud.config.server.environment.NativeEnvironmentRepository.findOne(NativeEnvironmentRepository.java:140)
The tipping point seems to be about 290KB or 10,000 lines. We have updated from Spring Boot 2.4.3 and Spring Cloud 2020.0.2 where we could retrieve the file successfully.
Example bootstrap config:
spring:
config:
additional-location: file:/run/secrets/auth.yml,file:/run/secrets/app-secrets.yml
application:
name: config-app
cloud:
inetutils:
ignored-interfaces:
- eth0
- eth1
- eth2
- eth3
- lo
consul:
discovery:
instanceId: ${spring.application.name}:${random.value}
host: consul
port: 8500
config:
name: large-config-file
discovery:
enabled: true
serviceId: Config-Server
username: xxxx
password: xxxx
fail-fast: true
retry:
initialInterval: 6000
maxInterval: 6000
maxAttempts: 10
profiles:
active: dev
server:
tomcat:
accesslog:
enabled: true
pattern: '%h %l %u %t "%r" %s %b %D %{X-B3-TraceId}i %{X-B3-SpanId}i'
directory: "/dev"
prefix: "stdout"
buffered: false
suffix: ""
file-date-format: ""
Comment From: mhalbritter
Hi!
If you'd like us to spend some time investigating, please take the time to provide a complete minimal sample (something that we can unzip or git clone, build, and deploy) that reproduces the problem.
Comment From: spring-projects-issues
If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.
Comment From: spring-projects-issues
Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.
Comment From: ruanych
I also encountered this problem just now. After investigation, I found that the place where the error is triggered lies in this configuration:
spring.application.name in application.yml.
If the input contains some characters, the exception will be thrown, I tested - ~ _ and they all cause the exception to be thrown.
I would venture to guess that the exception mentioned above is due to its configuration spring.application.name being config-app.
Comment From: mhalbritter
This does only happen if spring-cloud is used, right?
Comment From: ruanych
I am using springboot to build a web project and not using springcloud. I just tried it again, but this time with no errors!
I can't quickly locate where the problem is right now. If it reappears later, I will record more detailed content and try to give a reproduction plan.
Comment From: JalalSordo
imp having the same problem with spring boot 2.6.7 and 2021.0.2, I'm not sure if its related...
Comment From: philwebb
@JalalSordo Are you able to provide a sample application that replicates the problem?
Comment From: Madanor
We have the same issue. It looks like a off by one error in an array buffer if the file (or parts of it) have a specific size and the array is filled. I was able to isolate the file and will provide a sample. The issue happens during normal loading of the application.yml, no Spring cloud is required. It is reproducible in different projects with just dropping in the file. For example I used https://github.com/spring-guides/gs-spring-boot and put the file to initial/src/main/resources/application.yml, built and started. It also occurs with the latest Spring Boot 2.6.7.
The sample is "anonymized" because its stems a corporate project. The yaml is not valid (duplicate keys and ugly) but that doesn't matter for the issue because the exception occurs before the validity check.
Adding or removing a single character will not trigger the exception, it has to be the exact size (8732 Bytes). I also tried to build a sample with lots of comments in the file (lines starting with #) and the same file size, but that did not trigger the issue. So it is not the file size alone, but the amount of actual config parameters seems to be important. On the other hand the issue is still triggered when somewhere a newline is added and a character from a comment is removed or similar things, as long as the file size stays the same.
Comment From: wilkinsona
Thanks very much, @Madanor.
Using that application.yml file and SnakeYAML 1.29 (the default version in Spring Boot 2.6), I can reproduce the failure with the following test:
@Test
void arrayIndexOutOfBounds() throws Exception {
new Yaml().load(new FileReader(new File("application.yml")));
}
There's no Spring Boot code involved here so it appears to be a SnakeYAML bug.
Interestingly, the problem does not occur with SnakeYAML 1.28 (the default version in Spring Boot 2.5) or 1.30 (the default version in Spring Boot 2.7) so it appears to have been a regression in 1.29 that has been fixed in 1.30. If you're using Spring Boot 2.6.x and are affected by this problem, I'd recommend manually upgrading to SnakeYAML 1.30 using the snakeyaml.version property.
Comment From: Madanor
Are you planning to do a patch release for 2.6.x including SnakeYAML 1.30 at some point in the future? I would then refrain from setting the version manually in our pom structure, since the chance of the issue to occur is low and the workaround - changing the file size - is easy, and explicitly set versions require maintenance and tend to be forgotten during upgrades.
Comment From: wilkinsona
We are not. Our general rule is that we only pick up patch releases in Spring Boot patch releases. We consider SnakeYAML 1.30 to be a new feature release so it won't be included in Spring Boot 2.6.x.