This is regarding [spring-boot-maven-plugin]
Hello spring-boot-maven-plugin team,
I am here to report what seems to be an issue.
Currently using the latest 2.3.1
I am seeing the success from Maven, and the first run always is a success. However, on following runs, it is failing, because the previous maven run is blocking/hanging/holding. Long after I run the maven command, if I lsof -i tcp: 7654, I can still see the process there, and a ps -ef | grep shows /Library/Java/.../.../Contents/Home/bin/java -Dspring.application.admin.enabled=true -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=7654 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=127.0.0.1 -cp /Users/...
Is I manually kill the process using the port, this get unlocked, and I can do the run one more time, until it is blocked again. I tried to reproduce on a very small sample to share, however, it seems simple projects are not impacted. My current one is a maven multimode, where the structure is a parent with some common non springboot modules, with other springboot micro services. I want to run the (pre-integration-test) and (post-integration-test) on only one of them. But the issue mentioned is reproducible 100%.
It seems there is an issue where spring-boot-maven-plugin:2.3.1.RELEASE:stop (post-integration-test) can't take care of killing the process.
</environmentVariables>
<jvmArguments>
-Dspring.application.admin.enabled=true
</jvmArguments>
</configuration>
<executions>
<execution>
<id>pre-integration-test</id>
<goals>
<goal>start</goal>
</goals>
<configuration>
<jmxPort>7654</jmxPort>
</configuration>
</execution>
<execution>
<id>post-integration-test</id>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
Comment From: patpatpat123
It seems it is the same issue as this case here, after research on the web https://stackoverflow.com/questions/54081163/spring-boot-with-spring-boot-maven-plug-in-unable-to-restart-the-application-s
Comment From: wilkinsona
Thanks for the report. The problem is with your custom configuration. You have configured the start goal to use port 7654 to connect to the app but left the stop goal using the default. As a result, the stop goal believes that the application has already been stopped as it cannot connect.
If you want to use a custom for start and stop, you may want to consider configuring it once in the plugin's configuration rather than in each execution's configuration:
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<jmxPort>7654</jmxPort>
</configuration>
<executions>
<execution>
<id>pre-integration-test</id>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>post-integration-test</id>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
I'm going to keep this issue open as I think we could improve the documentation to note the important of the start and stop goals using the same jmxPort. We may also be able to do something when Maven's running to detect the misconfiguration.
Comment From: patpatpat123
Hello Andy,
Much appreciated for your detailed answer. I followed your step, and a bit of a bad news, it is still hanging. (I am not challenging, just relating a fact. Again, thank you for looking into this)
So, I moved the property "up":
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<wait>1000</wait>
<maxAttempts>180</maxAttempts>
<jmxPort>7654</jmxPort>
<environmentVariables>
...
</environmentVariables>
<jvmArguments>
-Dspring.application.admin.enabled=true
</jvmArguments>
</configuration>
<executions>
<execution>
<id>pre-integration-test</id>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>post-integration-test</id>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
And after the execution, the process is still hang. I can still see the process hanging on the port 7654, as well as the process being present after a ps -ef | grep Maybe I am doing something wrong? Thank you again @wilkinsona
Comment From: wilkinsona
Unfortunately, it's hard to tell if you're doing something wrong from only a snippet of XML. You could run Maven with -X and check the output to see if you can spot the problem. Alternatively, if you would like us to spend some more time investigating, please spend some time providing a complete yet minimal sample that reproduces the problem. You can share it with us by pushing it to a separate repository on GitHub or by zipping it up and attaching it to this issue.
Comment From: patpatpat123
I understand, make sense.
So first, the debug log. As you can imagine, the debug log is quite consequent. I am pasting the bit I believe can help, but if you are looking for anything in particular, please feel free to let me know.
[INFO] --- spring-boot-maven-plugin:2.3.1.RELEASE:start (pre-integration-test) @ ---
[DEBUG] Configuring mojo org.springframework.boot:spring-boot-maven-plugin:2.3.1.RELEASE:start from plugin realm ClassRealm[plugin>org.springframework.boot:spring-boot-maven-plugin:2.3.1.RELEASE, parent: jdk.internal.loader.ClassLoaders$AppClassLoader@2c13da15]
[DEBUG] Configuring mojo 'org.springframework.boot:spring-boot-maven-plugin:2.3.1.RELEASE:start' with basic configurator -->
[DEBUG] (f) addResources = false
[DEBUG] (f) agents = []
[DEBUG] (f) classesDirectory = …/target/classes
[DEBUG] (f) directories = []
[DEBUG] (f) environmentVariables = { … }
[DEBUG] (f) excludes = []
[DEBUG] (f) folders = []
[DEBUG] (f) fork = true
[DEBUG] (f) includes = []
[DEBUG] (f) jmxPort = 7654
[DEBUG] (f) jvmArguments = -Dspring.application.admin.enabled=true
[DEBUG] (f) maxAttempts = 180
[DEBUG] (f) profiles = []
[DEBUG] (f) project = MavenProject: … @ …/pom.xml
[DEBUG] (f) session = org.apache.maven.execution.MavenSession@183ade54
[DEBUG] (f) skip = false
[DEBUG] (f) useTestClasspath = false
[DEBUG] (f) wait = 1000
[DEBUG] -- end configuration --
[INFO] Attaching agents: []
[DEBUG] JVM argument(s): -Dspring.application.admin.enabled=true -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=7654 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=127.0.0.1
[DEBUG] Classpath for forked process: [If I use ps -ef | grep, this is exactly what I see here when the process hang]
[DEBUG] Application argument(s): --spring.application.admin.enabled=true --spring.application.admin.jmx-name=org.springframework.boot:type=Admin,name=SpringApplication
[DEBUG] Environment variable(s): …
[DEBUG] Connecting to local MBeanServer at port 7654
[DEBUG] Waiting for spring application to start...
[DEBUG] MBean server at port 7654 is not up yet...
[DEBUG] Spring application is not ready yet, waiting 1000ms (attempt 1)
[DEBUG] Connected to local MBeanServer at port 7654
[DEBUG] Waiting for spring application to start...
[DEBUG] Spring application is not ready yet, waiting 1000ms (attempt 1)
At this point my boot app starts. It is starting no problem, and failsafe is kicking
[INFO] --- spring-boot-maven-plugin:2.3.1.RELEASE:stop (post-integration-test) @ … ---
[DEBUG] Configuring mojo org.springframework.boot:spring-boot-maven-plugin:2.3.1.RELEASE:stop from plugin realm ClassRealm[plugin>org.springframework.boot:spring-boot-maven-plugin:2.3.1.RELEASE, parent: jdk.internal.loader.ClassLoaders$AppClassLoader@2c13da15]
[DEBUG] Configuring mojo 'org.springframework.boot:spring-boot-maven-plugin:2.3.1.RELEASE:stop' with basic configurator -->
[DEBUG] (f) jmxPort = 7654
[DEBUG] (f) project = MavenProject: … @ …/pom.xml
[DEBUG] (f) skip = false
[DEBUG] -- end configuration --
[INFO] Stopping application...
07-20 19:54:32 INFO - [-127.0.0.1] gistrar$SpringApplicationAdmin : [, ] Application shutdown requested.
07-20 19:54:32 INFO - [-127.0.0.1] .s.s.c.ThreadPoolTaskScheduler : [, ] Shutting down ExecutorService
[INFO]
[INFO] --- maven-failsafe-plugin:3.0.0-M5:verify (default) @ … ---
Is this helpful?
By the way, is it possible to have a debug log in stop to say the opposite of [DEBUG] Connected to local MBeanServer at port 7654 ?
Something like
[DEBUG] Disconnected from local MBeanServer at port 7654 successfully.
or
[DEBUG] Failed to disconnect from local MBeanServer at port 7654.
Thank you for your help.
Comment From: wilkinsona
The debug output was intended to help you to diagnose the problem. As I said above, if you would like us to spend some more time investigating, please spend some time providing a complete yet minimal sample that reproduces the problem. I can see from the debug output that your application has been stopped. If the JVM has not exited then there must be one or more non-daemon threads that are still running and are keeping the JVM alive. That suggests that there's a problem with your application or its configuration as it is starting a thread that isn't cleaned up when the application context is closed. A sample that reproduces the problem would allow us to identify the cause.
Comment From: snicoll
As discovered in #19048 I think we should use this issue to strengthen the fact that the default port could already be taken. This seems to be case more often on Windows.
Comment From: italktothewind
I had this problem and I fix it using an older version of the plugin (2.1.6.RELEASE). Luckily it works with the latest Spring Boot version (2.7.0)
Comment From: wilkinsona
@italktothewind Thanks for trying to help, but mixing versions like that is unsupported and may lead to unexpected problems.
Comment From: PatrikKoskenniemi
I am experiencing this problem on an application I recently starting working on and I've managed to reproduce it on a sample app but still not on the clear why it happens, probably due to my limited of understanding of threads in the jvm. In order to reproduce you need to defined a different TaskScheduler. So we have one bean like this:
@Bean
public TaskScheduler taskScheduler() {
return new ConcurrentTaskScheduler();
}
And you need a Scheduled task similar to this:
@Scheduled(fixedRate = 1800000)
public void runScheduledTask() {
System.out.println("Running scheduledTask");
}
Then the plugin looks like this. Observe that if we were to run it with fork=false everything works perfectly but this is not an option in out case:
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>2.7.1</version>
<configuration>
<jvmArguments>
-Dspring.application.admin.enabled=true
</jvmArguments>
</configuration>
<executions>
<execution>
<id>pre-integration-test</id>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>post-integration-test</id>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
Then we just run mvn verify and the process on port 9001 never dies.
For a fully working example I created a repo, please see: https://github.com/PatrikKoskenniemi/SampleSchedulingProblem
We could just remove the ConcurrentTaskScheduler bean and make it work but it feels like I'm missing something here, maybe someone else knows why this occurs?
Comment From: snicoll
@PatrikKoskenniemi I don't understand the link between your comment and this issue. This issue being closed, please create a separate issue. My best guess is that the scheduler prevents the app to shutdown. I don't know if we have a way to force it to exit but it's worth researching.
Comment From: PatrikKoskenniemi
@snicoll From what I can see above, it never got resolved and actually closed due to the fact that it was not reproducible in a simple way. So the link between my comment and the issue is that I am providing the sample app needed to figure the problem out. In my head it makes sense to re-open this one, but I could obviously open a new issue if you rather have it that way? Both works for me.
Comment From: snicoll
@PatrikKoskenniemi I don't read the issue history that way. The reporter had configured different ports for the start and stop goal and the documentation was improved to warn users about this problem. "Hanging" is the only thing that vaguely matches your sample project. Irrespective of your problem being similar or not, we can't reopen a documentation issue that was shipped 2 years ago.