If two included dependencies contain a module with the same name and same version number (which is unlikely, but might happen) the bootJar task in its current default behavior will generate a .jar file with colliding entries.

For example, these two libraries by accident contain the same module (but which are completely unrelated) and have by accident the same version string. (Which in it self should be no problem)

dependencies {
...
    implementation "org.web3j:core:3.4.0"
    implementation "com.google.zxing:core:3.4.0"
}

But the default bootJar task:

bootJar {
    launchScript()
    enabled = true
    archiveBaseName.set('foo')
}

.. will build without error and the resulting .jar file might also be runnable, but will crash non-deterministically on runtime either with - java.lang.NoClassDefFoundError: com/google/zxing/WriterException
- or java.lang.NoClassDefFoundError: org/web3j/protocol/Web3j

depending in which order the two files where included in the .jar file.

Unziping the bootjar file results in a overwrite-existing-file-warning:

> unzip foo.jar                   
Archive:  foo.jar
   creating: META-INF/
  inflating: META-INF/MANIFEST.MF    
....
 extracting: BOOT-INF/lib/core-3.4.0.jar  
 extracting: BOOT-INF/lib/abi-3.4.0.jar  
 extracting: BOOT-INF/lib/crypto-3.4.0.jar  
...
 extracting: BOOT-INF/lib/javax.json-1.1.4.jar  
 extracting: BOOT-INF/lib/barcode4j-2.1.jar  
replace BOOT-INF/lib/core-3.4.0.jar? [y]es, [n]o, [A]ll, [N]one, [r]ename:

And listing its content shows that it includes the same file with two different file-sizes:

> unzip -l foo.jar | grep core-3.4
   244695  2020-04-16 15:38   BOOT-INF/lib/core-3.4.0.jar
   539901  2020-04-16 15:38   BOOT-INF/lib/core-3.4.0.jar

Improvement

One easy improvement would be to default to duplicatesStrategy(DuplicatesStrategy.FAIL) for the bootJar task. I think there is no valid use-case to include a colliding filename twice in the jar without notifying the developer.

Which will lead to 

* What went wrong:
Execution failed for task ':foo:bootJar'.
> Encountered duplicate path "BOOT-INF/lib/core-3.4.0.jar" during copy operation configured with DuplicatesStrategy.FAIL

Workaround

Our workaround (also based on https://github.com/spring-projects/spring-boot/issues/10778 ) is to include the group id of the included modules:

bootJar {
...
    rootSpec.filesMatching('**/*.jar', { jar ->
        // only rename jars which are going into the lib dir into the fat jar
        if (jar.path.startsWith("BOOT-INF/lib")) {
            // rename the destination from "module-version.jar" to "group-module-version.jar"
            // eg
            //   BOOT-INF/lib/core-3.4.0.jar
            //   BOOT-INF/lib/core-3.4.0.jar
            // becomes
            //   BOOT-INF/lib/org.web3j-core-3.4.0.jar
            //   BOOT-INF/lib/com.google.zxing-core-3.4.0.jar
            String groupId = jar.file.parentFile.parentFile.parentFile.parentFile.name
            jar.name = "$groupId-${jar.name}"
        }
    })

    // to play it save, set this from IGNORE to FAIL, so that gradle aborts the build if it tries to include
    // two files with the same name
    duplicatesStrategy(DuplicatesStrategy.FAIL)
}

Comment From: philwebb

It's interesting that Gradle's War task doesn't change the default DuplicationStrategy. There some discussion about that here.

I'm not sure if it's best that we align with the War task, or change ours. The current situation certainly isn't great.

Comment From: wilkinsona

This is a duplicate of https://github.com/spring-projects/spring-boot/issues/16241. That issue led to use deciding to remain aligned with the behaviour of Gradle's War task. Things have moved on a little since then as, with the work on layered jars, we now have the capability of accessing the dependency coordinates for some (and typically all) of the jars going into BOOT-INF/lib. As such, I think it's worth considering this one again.

Comment From: DanielWeigl

My thinking here is, that BootJar will always emit a .jar file, which is inherently a .zip compressed file (or?) -> ZIP on its own allows multiple entries with the same name to be added into one file. But most tools won't handle files with colliding entries correctly (ie. Linux ARK tool crashes, unzip asks if you want to replace an existing file, there was a android package signing issue which exploited this undefined behavior, etc).

Also the class-loader (at least the one from OpenJDK 1.8) only loads one of the included modules from the .jar file.

So in my opinion it never makes sense to have a colliding file/module in a .jar file - or? Then it would at least make sense to fail at building (i.e. duplicatesStrategy(DuplicatesStrategy.FAIL)) but also allow a developer to overwrite the behavior if really needed for what ever reasons.

Comment From: wilkinsona

@DanielWeigl I don't think I disagree with any of that, but I also like the fact that we're currently aligned with Gradle's default behaviour for both jar and war artifacts.

Comment From: DanielWeigl

im not very used to .war artifacts, does it make sense for them to contain colliding files/libs? Then its more of an upstream bug.

But I also dont really see a big issue to not align with .war logic in this regards, if it means to handle an error as early as possible.

It took us quite a long time to track down the issue, because it only occurred randomly and only after we updated a very unrelated library (which lead to include this dependencies with the exact same version string)

Comment From: DanielWeigl

Ah, I see there is already some progress in this regards upstream in gradle: eg. https://github.com/gradle/gradle/issues/10856

But accordingly to https://github.com/gradle/gradle/issues/10039#issuecomment-535577223 it should emit a warning (we are using gradle 6.3), but even ./gradlew build --warning-mode all does not show anything regards duplicate file names... so not sure if it also affects the war/jar task.

Demo project here: https://gitlab.com/DanielWeigl/duplicatestrategyfailingdemo

Comment From: wilkinsona

Thanks for the demo project. It produces the expected warning for me:

$ ./gradlew bootJar --warning-mode all

> Task :bootJar
Copying or archiving duplicate paths with the default duplicates strategy has been deprecated. This is scheduled to be removed in Gradle 7.0. Duplicate path: "BOOT-INF/lib/core-3.4.0.jar". Explicitly set the duplicates strategy to 'DuplicatesStrategy.INCLUDE' if you want to allow duplicate paths. Consult the upgrading guide for further information: https://docs.gradle.org/6.3/userguide/upgrading_version_5.html#implicit_duplicate_strategy_for_copy_or_archive_tasks_has_been_deprecated

BUILD SUCCESSFUL in 1s
3 actionable tasks: 3 executed

Comment From: DanielWeigl

Oh - was just trying again, and now i see it too, with a clean

#> ./gradlew clean bootJar --warning-mode all

> Task :bootJar
Copying or archiving duplicate paths with the default duplicates strategy has been deprecated. This is scheduled to be removed in Gradle 7.0. Duplicate path: "BOOT-INF/lib/core-3.4.0.jar". Explicitly set the duplicates strategy to 'DuplicatesStrategy.INCLUDE' if you want to allow duplicate paths. Consult the upgrading guide for further information: https://docs.gradle.org/6.3/userguide/upgrading_version_5.html#implicit_duplicate_strategy_for_copy_or_archive_tasks_has_been_deprecated

BUILD SUCCESSFUL in 2s
4 actionable tasks: 4 executed

#> ./gradlew bootJar --warning-mode all      

BUILD SUCCESSFUL in 1s
3 actionable tasks: 3 up-to-date

I probably first build it without warning-mode all and then did not realize, that the BootJar task got cached.... damn.

So, i guess we are in the clear here, and just need to wait for gradle 7. Thx for testing.

I mean - it will make the problem of colliding module-names more visible, but I guess the BootJar task will still need to provide a workaround on how to handle this situation.

Comment From: philwebb

We've decided to keep things aligned with the Gradle defaults for now. When Gradle 7 comes out, we'll pick up the duplication failure error for free.