Sorry to raise an issue about this, but I can't find any documentation that allows me to work around this:
Earlier this week, we were building and deploying spring-boot based docker images just fine, using gradle bootBuildImage --imageName=...
. Suddenly our deploys stopped working though. We had just merged support for reactor's BlockHound and got an error that came down to:
Caused by: java.lang.IllegalStateException: No compatible attachment provider is available
I then commented out the BlockHound.install()
but now our deploy still triggered an error:
Unrecognized option: -server
I then checked our build logs and noticed that the previous, working builds used:
> Running creator
[creator] ===> DETECTING
[creator] 5 of 16 buildpacks participating
[creator] paketo-buildpacks/bellsoft-liberica 2.13.0
[creator] paketo-buildpacks/executable-jar 2.1.1
[creator] paketo-buildpacks/apache-tomcat 1.5.0
[creator] paketo-buildpacks/dist-zip 1.4.0
[creator] paketo-buildpacks/spring-boot 2.5.0
And the new, failing ones use:
> Running creator
[creator] ===> DETECTING
[creator] 5 of 17 buildpacks participating
[creator] paketo-buildpacks/bellsoft-liberica 3.2.0
[creator] paketo-buildpacks/executable-jar 3.1.0
[creator] paketo-buildpacks/apache-tomcat 2.2.0
[creator] paketo-buildpacks/dist-zip 2.2.0
[creator] paketo-buildpacks/spring-boot 3.1.0
I'm now assuming that version bump is incompatible with our app? Is there some setting I can change to revert to the old buildpack? Or maybe force it to use a JDK inside (which is what the BlockHound error seems to be about)?
Comment From: jgrgt
In case anyone has the same issue:
Fixed it by falling back to a more manual approach:
Add a Dockerfile to your repository
FROM adoptopenjdk:8-jdk-hotspot as builder
WORKDIR application
ARG JAR_FILE=build/libs/app.jar
COPY ${JAR_FILE} application.jar
RUN java -Djarmode=layertools -jar application.jar extract
FROM adoptopenjdk:8-jdk-hotspot
WORKDIR application
COPY --from=builder application/spring-boot-loader/ ./
COPY --from=builder application/dependencies/ ./
COPY --from=builder application/snapshot-dependencies/ ./
COPY --from=builder application/application/ ./
ENTRYPOINT ["java", "org.springframework.boot.loader.JarLauncher"]
(Use jdk/jre as you prefer, I switch to jdk because I wanted BlockHound to work.)
Then configure your bootJar
task to use layers and use app.jar
as an archive name:
tasks.getByName<BootJar>("bootJar") {
archiveFileName.value("app.jar")
layered()
}
Now instead of running bootBuildImage
, run bootJar
, and build the docker image like this:
docker build . --tag=...
I like the fact I have more control this way. Builds changing dependencies behing my back is not something I like in my CI/CD flow.
Comment From: nebhale
@jgrgt I've just reproduced this and I'll work with the BlockHound team to find out what the story is.
Comment From: nebhale
@jgrgt Can you describe any other configuration you have around arguments passed to the build? Specifically do you have any idea where -server
might be coming from and where the error is actually coming out of?
Comment From: bsideup
There seems to be an issue in ByteBuddy, just using net.bytebuddy:byte-buddy-agent:1.10.14
and ByteBuddyAgent.install();
makes it fail the same way (cc @raphw).
Here is the reproducer by @nebhale:
https://github.com/nebhale/BOOT-23121
Once cloned, ./gradlew bootBuildImage --imageName=applications/gradle-demo && docker run applications/gradle-demo:latest
will the failure described in the issue.
However, adding net.java.dev.jna:jna-platform:5.6.0
enables the emulated attachment strategy and makes it start correctly.
Comment From: jgrgt
Thanks for looking into this!
@nebhale I checked our infra code and the -server
is in a JAVA_OPTS environment variable. Sorry! This used to work, but now no longer works with bootBuildImage
. But that can be changed on our side of course.
@bsideup I can confirm that adding implementation("net.java.dev.jna:jna-platform:5.6.0")
to my dependencies makes the BlockHound issue go away. Thx!
Comment From: nebhale
Argh, yeah -server
is a really weird option these days. A little background:
Previously we'd use a command like java -cp "${CLASSPATH}" ${JAVA_OPTS} <Main-Class>
to start an application. Unfortunately, we had multiple users experiencing issues getting flags, via ${JAVA_OPTS}
, reliably into the various Java processes that might run inside of a container. To solve this we decided to depend on two features built natively into modern JVMs:
${CLASSPATH}
is read from the environment as an alternative to-cp
${JAVA_TOOL_OPTIONS}
is read from the environment as an alternative to flags on the command line
The command line we create now is java <Main-Class>
and that's great, but unfortunately, it appears that not all flags are legal in ${JAVA_TOOL_OPTIONS}
. You can see an example of this locally on your machine:
➜ java -server -version
openjdk version "11.0.8" 2020-07-14 LTS
OpenJDK Runtime Environment (build 11.0.8+10-LTS)
OpenJDK 64-Bit Server VM (build 11.0.8+10-LTS, mixed mode)
➜ JAVA_TOOL_OPTIONS="-server" java -version
Picked up JAVA_TOOL_OPTIONS: -server
Unrecognized option: -server
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Given that the number of problems we solved by going from ${JAVA_OPTS}
to ${JAVA_TOOL_OPTIONS}
was so large, it's unlikely that we'll go back. In fact, -server
is ignored in 64-bit linux JVMs at this point (in lieu of directly configuring compiler flags). If it's critically important that you set -server
, Java 9 and above have yet another flag, ${JDK_JAVA_OPTIONS}
which will allow you to set -server
.
Comment From: raphw
However, adding
net.java.dev.jna:jna-platform:5.6.0
enables the emulated attachment strategy and makes it start correctly.
That's quite expected. If you are using a JRE and no full JDK, the virtual machine APIs are missing and it is impossible to attach. Unless of course, Byte Buddy can do the attachment itself what requires a native adapter which is JNA.
Comment From: bsideup
@raphw thanks for your answer!
I thought that, on Java 9+, BB works fine on JRE? I am pretty sure I've seen it running on JRE just fine, without JNA...
Comment From: raphw
Technically, there's no JRE anymore in Java 9. Some people do distribute 'reduced module JVMs' though and call them JRE. If the jdk.attch module is missing, attachment does no longer work, the same way as JDBC won't work without the java.sql module.
Note that the java.instrument module is also required and cannot be emulated.
Comment From: bsideup
@raphw hm, the module is indeed missing:
$ java --list-modules | grep attach
$
It seems that Liberica excludes the module and the emulation needs to be used.
Thanks for you help! 💯
Comment From: raphw
Liberica diatributes both a JDK and a "JRE" where the JRE includes the modules most similar to Java 8's JRE spec.
Comment From: jgrgt
@nebhale Thanks for the insight on the -server
flag. We do not require it, so we can drop it.
Together with the jna-platform
dependency we can now use bootBuildImage
again.
Comment From: scottfrederick
Closing this issue since it's clear that it is related to Paketo buildpacks and not something that can be addressed by Spring Boot.