Hi,

this PR gets us one step further into supporting JDK 16 (see #24402).

I'm at a stage where I have random failures on my machine that seem related to network errors & timeouts, so I'm both confident and desperate enough to share my first draft.

Essentially, the PR replaces the functionality of buildJavaHome with a new property toolchainVersion that controls whether or not an optional toolchain should be configured for compilation, javadoc and test runs.

I should note that the spring-boot-gradle-plugin has multiple problems. None of the current versions support JDK 16 and thus a lot of tests fail. Unfortunately, you can't provide an empty stream in GradleCompatibilityExtension.provideTestTemplateInvocationContexts. I had some hacky code that would workaround that but I figured that a simple exclude would suffice for the time being. E.g. I used the following:

./gradlew build -x :spring-boot-project:spring-boot-tools:spring-boot-gradle-plugin:test -x :spring-boot-project:spring-boot-tools:spring-boot-gradle-plugin:validatePlugins -PtoolchainVersion=16

As you can see, I excluded the validatePlugins task as well, which is failing due to https://github.com/gradle/gradle/issues/15538.

The big issue is something else though. With JDK 16 and more specifically JEP-396 we have strong encapsulation enabled by default. That breaks a lot of tests and libraries. Just to name a few:

  • Any AbstractServletWebServerFactoryTests due to the URL.factory field being reset.
  • CliTester for a similar reason
  • JsonbTesterTests due to a bug in Johnzon
  • A couple of tests related to LDAP because of https://github.com/spring-projects/spring-ldap/issues/570
  • ...

I decided to add --illegal-access=warn so we get warned, but at the same time have access allowed for the time being.

Feel free to test around with this PR. Again - I didn't have a single test run today that was green, but all test failures were random and successful when run in isolation.

Next steps

If this PR is merged eventually, I would tackle the actual pipeline. I thought this might be a bit much for this PR, given that I need to add a secondary JDK to the CI images. (I can hopefully reuse some logic though that I build in the past for Boot already)

Let me know what you think. Christoph

Comment From: wilkinsona

Thanks very much, @dreis2211. Here's a build scan for the proposed changes: https://ge.spring.io/s/ovgtv4h7lapkc.

The failures in the CLI appear to be due to Groovy not supported Java 16 byte code. The JarCommandIT tests appear to be failing due to Groovy's version of ASM not supporting Java 16 byte code. It's not apparent from the test's output, but running in my IDE revealed the following output from the jar command:

Caused by: java.lang.IllegalArgumentException: Unsupported class file major version 60
    at groovyjarjarasm.asm.ClassReader.<init>(ClassReader.java:196)
    at groovyjarjarasm.asm.ClassReader.<init>(ClassReader.java:177)
    at groovyjarjarasm.asm.ClassReader.<init>(ClassReader.java:163)
    at groovyjarjarasm.asm.ClassReader.<init>(ClassReader.java:284)
    at org.codehaus.groovy.ast.decompiled.AsmDecompiler.parseClass(AsmDecompiler.java:81)
    at org.codehaus.groovy.control.ClassNodeResolver.findDecompiled(ClassNodeResolver.java:251)
    at org.codehaus.groovy.control.ClassNodeResolver.tryAsLoaderClassOrScript(ClassNodeResolver.java:189)
    at org.codehaus.groovy.control.ClassNodeResolver.findClassNode(ClassNodeResolver.java:169)
    at org.codehaus.groovy.control.ClassNodeResolver.resolveName(ClassNodeResolver.java:125)
    at org.codehaus.groovy.ast.decompiled.AsmReferenceResolver.resolveClassNullable(AsmReferenceResolver.java:57)
    at org.codehaus.groovy.ast.decompiled.Annotations.createAnnotationNode(Annotations.java:40)
    at org.codehaus.groovy.ast.decompiled.DecompiledClassNode.addAnnotations(DecompiledClassNode.java:268)
    at org.codehaus.groovy.ast.decompiled.DecompiledClassNode.lazyInitSupers(DecompiledClassNode.java:184)
    at org.codehaus.groovy.ast.decompiled.DecompiledClassNode.getInterfaces(DecompiledClassNode.java:98)
    at org.codehaus.groovy.ast.ClassNode.getInterfaces(ClassNode.java:375)
    at org.codehaus.groovy.ast.ClassNode.getAllInterfaces(ClassNode.java:438)
    at org.codehaus.groovy.ast.ClassNode.getAllInterfaces(ClassNode.java:440)
    at org.codehaus.groovy.ast.ClassNode.getAllInterfaces(ClassNode.java:430)
    at org.codehaus.groovy.control.ResolveVisitor.setRedirect(ResolveVisitor.java:526)
    at org.codehaus.groovy.control.ResolveVisitor.resolveNestedClass(ResolveVisitor.java:485)
    at org.codehaus.groovy.control.ResolveVisitor.resolve(ResolveVisitor.java:462)
    at org.codehaus.groovy.control.ResolveVisitor.resolve(ResolveVisitor.java:428)
    at org.codehaus.groovy.control.ResolveVisitor.resolveOrFail(ResolveVisitor.java:343)
    at org.codehaus.groovy.control.ResolveVisitor.visitAnnotations(ResolveVisitor.java:1330)
    at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitClass(ClassCodeVisitorSupport.java:51)
    at org.codehaus.groovy.control.ResolveVisitor.visitClass(ResolveVisitor.java:1465)
    at org.codehaus.groovy.control.ResolveVisitor.startResolving(ResolveVisitor.java:230)
    at org.codehaus.groovy.control.CompilationUnit$13.call(CompilationUnit.java:700)
    at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:965)
    ... 17 more

Support for Java 16 has landed in ASM but it has not yet been released. Framework is avoiding this problem by patching its own version of ASM to pull in the latest changes prior to them being released. The ASM release with Java 16 is expected to be "at the same time the JDK 16 will be released" so we may have to live with this for a while by skipping those tests on Java 16.

Comment From: dreis2211

@wilkinsona Thanks. I'm wondering why the test you mentioned didn't fail on my machine.

But seeing no indication of the toolchain, I think I will bring back a custom value.

Comment From: wilkinsona

The ASM release with Java 16 is expected to be "at the same time the JDK 16 will be released" so we may have to live with this for a while by skipping those tests on Java 16.

Things seem to have happened more quickly on the ASM side than the ticket linked to above suggested. According to https://asm.ow2.io/versions.html, ASM 9.0 – that includes Java 16 support – was released in September.

Comment From: wilkinsona

But seeing no indication of the toolchain, I think I will bring back a custom value.

I couldn't see any indication either. I raised it with the Gradle team and they confirmed it's not yet surfaced in the builds scans. They suggested using custom values for now.

Comment From: wilkinsona

I'm wondering why the test you mentioned didn't fail on my machine.

Me too. Turns out there's a bug in CommandLineInvoker. If there's more than one zip in the build output it'll use the first one it finds which may not be the right one. As a result, the tests were using 2.4's CLI with Groovy 2.5.x. They pass with 2.5's CLI and Groovy 3.0.x.

Comment From: dreis2211

@wilkinsona Thanks for checking that. I indeed did some clean runs in between.

I added the custom toolchain value to the scans. Let me know if you miss anything on this PR still.

Comment From: dreis2211

@philwebb I saw you changed the title, but I'm personally not comfortable with saying "Support Java 16" just yet. The issues over at https://github.com/spring-projects/spring-ldap/issues/570 and the Gradle Plugin not being supported at all maybe gives the wrong impression to users that read the changelog and see that. In the end, it's more "Support Java 16 builds" and one step further into supporting Java 16.

Comment From: wilkinsona

We're happy with stating that Spring Boot itself supports Java 16, even if that's not true across the board. For example, our Java 9 support excluded Cassandra for several months. If necessary, we can do the same with Spring LDAP and Gradle. The latter's really not too bad as users can still use Java 16 at runtime as long as they run Gradle itself on an earlier JVM.

Comment From: dreis2211

Fair enough...

Comment From: dreis2211

@wilkinsona @snicoll I've pushed a revised version that extracts the toolchain configuration into its own plugin/extension. That felt cleaner overall, but let me know what you think of it.

The custom plugin + extension has the possibility to set a maximum compatible java version and disables the toolchain based tasks if it doesn't match (see the build.gradle of spring-boot-cli for an example) The positive side-effect of this is that there is no need for custom task exclusions via ./gradlew build -x excludedTask anymore, which will be benefitial for the overall pipeline complexity.

Comment From: wilkinsona

This is great. Thanks very much, @dreis2211. I particularly like the new approach to configuring the toolchain stuff via the plugin and extension.

Comment From: wilkinsona

Here is a scan for a 2.4.x build using a Java 16 toolchain.