We are trying to do integration testing by loading the Spring Boot environment, injecting some dependencies into our test classes via @Autowired, etc. We're running into issues where our tests are competing with the CommandLineRunner that starts when the SB environment loads.

Comment From: philwebb

Could you use profiles to exclude the CommandLineRunner beans from your tests?

Comment From: berlin-ab

Is it possible to say @NotProfile("test")?

Comment From: philwebb

You can use ! as a prefix. e.g. @Profile("!test"). See https://jira.spring.io/browse/SPR-8728 for background.

Comment From: berlin-ab

@philwebb What does Spring Boot do for the embedded tomcat instance when running integration tests?

Comment From: philwebb

@berlin-ab It can start it up fully if needed. Our internal integration tests also set server.port to 0 so it runs on a random port.

See https://github.com/spring-projects/spring-boot/blob/master/spring-boot-samples/spring-boot-sample-tomcat/src/test/java/sample/tomcat/SampleTomcatApplicationTests.java

Comment From: berlin-ab

@philwebb I thought there was something about it not starting the container if MockMVC is in use.

Comment From: snicoll

You need to add @WebAppConfiguration. Look at the example that Phil gave.

Comment From: berlin-ab

@snicoll @philwebb When I'm not running a web application, I'd like to have the same behavior for a command line application.

Comment From: dsyer

What's the "same behaviour"? The same as what? What did you mean by "competing" in the original question? Does the CommandLineRunner do something in a background thread?

Comment From: berlin-ab

@dsyer yes, we start up a new thread.

Comment From: dsyer

I'd have to see the code to really understand why you did that. If the app depends on something happening in that thread, then there's a race condition at runtime whether it's a test or a production environment, right? So I would expect you'd have to deal with that somehow in any case.

Comment From: berlin-ab

No, we would not have this race condition in Production.

Here's what we have: CommandLineRunner -> Thing -> Thread -> RepeatedTask

We want to be able to test the RepeatedTask, but we use:

@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes=Application.class)

so we can inject @Components into RepeatedTask, which causes the CommandLineRunner to start and creates a race condition between CommandLineRunner -> Thing -> Thread -> RepeatedTask ) vs RepeatedTask.

Comment From: dsyer

Why not create a standalone ApplicationContext that just creates a RepeatedTask and all its dependencies, and just test that then?

Comment From: ravishankars5355

Hi,

I have a question. Requesting your help. We have a spring boot application and i have written integration tests without Mocks with TestRestTemplate and @SpringbootTest.

So on local machine, when i execute the tests, they execute fine as i have given MyApplication.class inside @Springboottest It will startup the spring application context and execute the tests.

Till here everything is fine.

But we deploy this application on different test environments like qa,e2e,staging and then on production. So we have to execute the Jenkins Job for my integration tests against the above environments as an acceptance tests.

My Question over here is : - When i execute these tests on jenkins, the tests get executed on a Jenkins Slave machine(which is picked randomly among the available executors) and it will hit the end points (either qa or e2e or staging or production end points) and send rest requests and get the responses and validate. But the tests start up the application context on the jenkins slave and loads on a random port and will be available on the jenkins slave machine till the tests finish though i am not at all interacting with the application context( as i am hitting external test end points). Is there any option not to load the spring application context when i am trying to run tests against real test server and to load the application context when testing on local?

Please help.. I am kind of new to spring boot and got stuck here.

Thank you very much in advance

Thanks, Ravi Shankar

Comment From: Sourabh25

I also have the same scenario. Any help would be highly appreciable. Thanks

Comment From: pavankumarchaitanya

Try printing the name of the active profile under which the application is running in the tests. If they're different when running locally vs when running on the Jenkins machines then you're one step closer to fixing the issue.

Comment From: ravishankars5355

@pavankumarchaitanya :- its all the same on both local machine and the jenkins slave machine. But is there a way to stop the container when not executing the tests on local environment. I have posted questions on many forums and there seems to be no answer yet. Requesting help

Comment From: mohamnag

I also have this same problem and by looking at this question on SO, I see we are not the only ones. Using the profile MAY be a solution but I see following two problems: * who guarantees that nobody sets the profile in runtime and therefore prevents parts of code to run? * in general I dont feel good when the code should be changed just for the sake of tests

I think this is an straight forward feature request, SpringRunner shall allow postponing context start until later. There are many reasons, for example my app sends specific messages over Kafka on startup and I want to test exactly that functionality. Or another app processes some file, store results in DB and shuts down and thats exactly what we want to test. I can imaging an elegant solution is to have a config attribute (maybe as part of @SpringBootTest) that flags not to start the context and then have a junit rule or similar that by calling a method on it, it boots the context.

Is that something feasible?

Comment From: sdoeringNew

I have to mock several external resources but I can't mock them because the CommandLineRunner already runs before the test starts. And guess what... The test fails to start because the CommandLineRunner fails as the external resources haven't been mocked.

That is so irrational. The SpringBootTest shall enable testing with an initialized application context but the application to test runs before the tests.

Comment From: wilkinsona

I have to mock several external resources but I can't mock them because the CommandLineRunner already runs before the test starts.

That shouldn't prevent you from mocking them. It prevents you from mocking them using @MockBean (as you can't set an expectations on the mocks before they're called), but you can use a @TestConfiguration class and some @Bean methods that return mocks instead.

Comment From: sdoeringNew

I have to mock several external resources but I can't mock them because the CommandLineRunner already runs before the test starts.

That shouldn't prevent you from mocking them. It prevents you from mocking them using @MockBean (as you can't set an expectations on the mocks before they're called), but you can use a @TestConfiguration class and some @Bean methods that return mocks instead.

Unfortunately I can't do that. I don't have to mock Beans. I have to mock external web services, e.g. with WireMock, or other TCP connections.

Comment From: wilkinsona

That should still be possible if you set up the mock via a @Configuration class, InitializingBean, @PostConstruct method, etc. All of those will be called before any CommandLineRunner beans which are not invoked until the application context has been refreshed.

Comment From: sdoeringNew

Ok. I've just tried it and it works. It is unnatural and ugly to: - define the stubs, mocks and test variables outside of the test method - not calling the tested method in the test method - only do assertions in the test method

And furthermore more complex for various tests as each test method needs its own test class.

...but at least it does the job.

Thanks.

Comment From: wilkinsona

If you want to do it all in the test method, then @SpringBootTest is the wrong tool for the job. Instead, I would recommend that you start your application yourself in your @Test method.

Comment From: sdoeringNew

Then I loose a convenient way of overwriting properties. :disappointed: But maybe it is worth it to get rid of all the other disadvantages.

Comment From: sdoeringNew

I was able to mock all necessary external services before the CommandLineRunner started.

Now there is another problem. As I use a @Scheduled task the application is not stopping if the CommandLineRunner is done.

@AllArgsConstructor
public class CommandRunner implements CommandLineRunner {
    private final ConfigurableApplicationContext applicationContext;

    @Override
    public void run(final String[] args) {
        // long running task
        // ..

        // stop the application
        applicationContext.close();
    }
}

But with that the SpringBootTest does not work anymore because it stops after the CommandLineRunner run the first time - before the test.

How can I workaround this issue?

Edit: I tried to Mock the ConfigurableApplicationContext without success.

@MockBean
ConfigurableApplicationContext configurableApplicationContext;

Comment From: wilkinsona

@sdoeringNew If code that your test is executing closes the context and you're trying to use @SpringBootTest, you'll need to mark your test class with @DirtiesContext so that the test framework knows that the context cannot be reused.

If you have any further questions, please follow up on Stack Overflow or Gitter. As mentioned in the guidelines for contributing, we prefer to use GitHub issues only for bugs and enhancements.

Comment From: sdoeringNew

The test method itself does not execute any code, as the CommandLineRunner - that runs before the test - already closes the Context.

Adding @DirtiesContext will not solve the problem.

So we are still coming back to this issue:

Is there a reason the CommandLineRunner runs before the test? If not please consider deactivating running CommandLineRunner in SpringBootTest at all.

Comment From: wilkinsona

Is there a reason the CommandLineRunner runs before the test?

Yes. A @SpringBootTest starts a SpringApplication for you and part of the contract of SpringApplication is that it will run any CommandLineRunner beans found in the context. We cannot break that contract by default as it is the expected and documented behaviour. If you do not want your CommandLineRunner to be included in a particular test then you could use profiles or you could consider if @SpringBootTest is the right tool for the job. I'd also consider whether your CommandLineRunner should be closing the application context or if that should be done elsewhere.

As I said above, if you have any further questions, please follow up on Stack Overflow or Gitter. As mentioned in the guidelines for contributing, we prefer to use GitHub issues only for bugs and enhancements.

Comment From: sdoeringNew

How about adding a new Test-Context for that case? That would not be the default and therefore not breaking the contract?

Using profiles is still no good solution as stated before.

I might be mistaken, but I see a CommandLineRunner application as a tool that runs once and then stops. Nobody combines a WebApplication and a CommandLineRunner. Or add three CommandLineRunner to one application. (Am I mistaken?)

If a SpringBootTest is added to an CommandLineRunner application than it shall test this CommandLineRunner and nothing else.

Comment From: wilkinsona

Yes, I think you are mistaken.

There's nothing unusual about using a CommandLineRunner in a web application. It's simply a way for a bean to be invoked with the application's arguments. It can be used in any type of application that has this requirement.

There's also nothing unusual about defining multiple CommandLineRunner beans. From the class's javadoc:

Multiple CommandLineRunner beans can be defined within the same application context and can be ordered using the Ordered or @Order annotation.

Comment From: sdoeringNew

I understand.

So what about the suggestion of adding a new Test-Context which does not break the contract of executing the CommandLineRunner? Or at least another flag in the SpringBootContext with 'runCommandLineRunner' which defaults to true.

The demand exists: https://stackoverflow.com/questions/29344313

Comment From: wilkinsona

Sorry, but I don't think we want to do that.

@SpringBootTest is intended to test the complete application, and that includes any CommandLineRunner beans. Excluding beans from a @SpringBootTest is discouraged as it means that your integration test is not testing the entirety of your application.

If you are happy to accept that risk, please use one of the existing mechanisms for excluding a bean. In addition to using profiles, several other alternatives are described in the answers to the Stack Overflow question to which you linked above.

Comment From: rwxguo

I suppose someone wants to use CommandLineRunner as the application entry of Standalone Application / Console Application. (As some website teachs to do like this)

If it's your case, Here is a more reasonable way to implement.

As @wilkinsona said, CommandLineRunner is part of Springboot, its purpose is probably to provide access to application arguments. Although the behavior of CommandLineRunner is similair to Main function of general console application, but working with Springboot, It's not a good idea.