When the Maven spring-boot:build-image
goal or Gradle bootBuildImage
task is run more than once concurrently against the same Docker daemon, builds can fail with an error like
Docker API call to 'localhost/v1.24/images/pack.local/builder/zuslfxipuv:latest?force=1' failed with status code 500 "Internal Server Error" and message "unrecognized image ID sha256:xxxxx"`
Comment From: scottfrederick
When the Spring Boot plugins CNB integration builds an image, it follows a series of steps:
- pull the builder image (by default
paketobuildpacks/builder:base
) - clone the builder image to create an ephemeral builder image with a name like
pack.local/builder/[random string]
- embellish the ephemeral builder image with any environment variables or buildpacks specified in the build configuration along with metadata like the Spring Boot plugin version
- create a container from the ephemeral builder image and run the CNB lifecycle in the container
- delete the ephemeral builder container and image
If Spring Boot runs two builds in parallel, then two ephemeral builder images get created. If the base image, build parameters, and image metadata are identical then internally the Docker daemon creates one image with two tags instead of creating two unique images. Listing images in the Docker daemon while the build is running might look like this (note that both ephemeral images have the same IMAGE ID
:
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
paketobuildpacks/builder base 06d50806f1c2 41 years ago 756MB
pack.local/builder/pcspywwker latest 316c12b7dd8d 41 years ago 756MB
pack.local/builder/sstditlopy latest 316c12b7dd8d 41 years ago 756MB
This can result in a race condition inside the Docker daemon when Spring Boot asks Docker to delete the two ephemeral images at exactly the same time. Under just the right conditions the daemon will successfully delete the single image referred to by one tag and throw an unrecognized image ID sha256:316c12b7dd8d...
error when it tries to delete the second tag because the underlying image has already been deleted.
Comment From: dokaspar
Hi @scottfrederick
Does this imply that the only solution to the problem is upgrading to at least version 2.4.11?
Or are there any other workarounds, such as running Gradle with the --no-parallel
flag?
Regards
Dominik
Comment From: scottfrederick
Does this imply that the only solution to the problem is upgrading to at least version 2.4.11?
Yes, that's the only way to be sure you won't encounter this problem.
Also note that Spring Boot 2.4.x and earlier are out of open-source support. You should upgrade to a supported version of Spring Boot.
Or are there any other workarounds, such as running Gradle with the --no-parallel flag?
Gradle's --no-parallel
flag might work if you have multiple images being built from the same Gradle build. It won't help if you are running multiple Maven or Gradle builds against the same Docker daemon in separate processes, as is common in CI systems.
Comment From: dokaspar
Thanks a lot for the quick response and the clarifications!