SpringBoot ServiceLoader do not work in packaged Spring Boot apps

(StackOverflow post: https://stackoverflow.com/questions/63414862/serviceloader-do-not-work-in-packaged-spring-boot-apps)

I noticed that Java's ServiceLoader mechanism doesn't work in packaged Spring Boot apps.

Background

I've tried to use javax.script.ScriptEngineManager which relies on ServiceLoaders. I was able to successfully launch the app from the IDE but not from the command line.

Repro

// build.gradle
plugins {
    id 'java'
    id 'org.springframework.boot' version '2.3.2.RELEASE'
    id 'io.spring.dependency-management' version '1.0.9.RELEASE'
}

tasks.withType(JavaCompile) {
    options.release.set(11) // required Gradle >= 6.6
}

repositories {
    mavenCentral()
}

dependencies {
    implementation 'org.springframework.boot:spring-boot-starter-web'
    implementation 'org.python:jython-slim:2.7.2'
}

// Main.java
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

import javax.script.ScriptEngineManager;
import java.util.Objects;

@SpringBootApplication
public class Main {
    public static void main(String[] args) {
        SpringApplication.run(Main.class, args);
        var engine = new ScriptEngineManager().getEngineByName("python");
        Objects.requireNonNull(engine);
        System.out.println("success");
    }
}

It is possible to launch this from the IDE (IntelliJ in my case) but not via the command line:

gradle bootJar && java -jar build/libs/XXX.jar

Temporary workaround

Instead of using the ScriptEngineManager it is possible to directly use Jython's ScriptEngine implementation:

  var engine = new org.python.jsr223.PyScriptEngineFactory().getScriptEngine();

Comment From: wilkinsona

To avoid duplicate effort, I'm going to close this in favour of the Stack Overflow question.