It appears that @DynamicPropertySource cannot be used on a test superclass. See https://twitter.com/aheritier/status/1311721275811917824

Comment From: aheritier

Effectively @philwebb

I am using Spring Boot 2.3.4.RELEASE with testcontainers 1.15.0-rc2 (but I don't think it has any impact here.)

My tests are using JUnit 5 and I was creating a parent/base class for all my tests suites using the container as described in: https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#howto-testcontainers

With such approach only the tests executed in the first test class are working and the next ones are failing.

@Container and @DynamicPropertySource in a parent class

Only the the 1st test class is succeeding because the configuration isn't updated for others tests

2020-10-01 19:21:13.966 INFO 52376 --- [           main] c.c.s.analytics.ElasticsearchTestCase    : spring.elasticsearch.rest.uris:localhost:32848
19:21:02.888 [main] INFO org.testcontainers.containers.wait.strategy.HttpWaitStrategy - /mystifying_joliot: Waiting for 120 seconds for URL: http://localhost:32848/
2020-10-01 19:21:26.781  INFO 52376 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /boring_ganguly: Waiting for 120 seconds for URL: http://localhost:32850/
2020-10-01 19:21:37.942  INFO 52376 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /condescending_galileo: Waiting for 120 seconds for URL: http://localhost:32852/
2020-10-01 19:21:49.065  INFO 52376 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /upbeat_neumann: Waiting for 120 seconds for URL: http://localhost:32854/
2020-10-01 19:22:00.284  INFO 52376 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /compassionate_lamarr: Waiting for 120 seconds for URL: http://localhost:32856/
2020-10-01 19:22:11.489  INFO 52376 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /jolly_keller: Waiting for 120 seconds for URL: http://localhost:32858/

@Container in a parent class and @DynamicPropertySource in every test class

It's ok

2020-10-01 19:27:08.172 INFO 55726 --- [           main] c.c.s.analytics.ElasticsearchTestCase    : spring.elasticsearch.rest.uris:localhost:32861
19:26:58.269 [main] INFO org.testcontainers.containers.wait.strategy.HttpWaitStrategy - /elegant_wu: Waiting for 120 seconds for URL: http://localhost:32861/
2020-10-01 19:27:31.272 INFO 55726 --- [           main] c.c.s.analytics.ElasticsearchTestCase    : spring.elasticsearch.rest.uris:localhost:32863
2020-10-01 19:27:22.046  INFO 55726 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /pensive_nightingale: Waiting for 120 seconds for URL: http://localhost:32863/
2020-10-01 19:27:49.474 INFO 55726 --- [           main] c.c.s.analytics.ElasticsearchTestCase    : spring.elasticsearch.rest.uris:localhost:32865
2020-10-01 19:27:40.219  INFO 55726 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /epic_haslett: Waiting for 120 seconds for URL: http://localhost:32865/
2020-10-01 19:28:07.399 INFO 55726 --- [           main] c.c.s.analytics.ElasticsearchTestCase    : spring.elasticsearch.rest.uris:localhost:32867
2020-10-01 19:27:57.138  INFO 55726 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /angry_gagarin: Waiting for 120 seconds for URL: http://localhost:32867/
2020-10-01 19:28:24.925 INFO 55726 --- [           main] c.c.s.analytics.ElasticsearchTestCase    : spring.elasticsearch.rest.uris:localhost:32869
2020-10-01 19:28:15.596  INFO 55726 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /competent_jennings: Waiting for 120 seconds for URL: http://localhost:32869/
2020-10-01 19:28:43.334 INFO 55726 --- [           main] c.c.s.analytics.ElasticsearchTestCase    : spring.elasticsearch.rest.uris:localhost:32871
2020-10-01 19:28:34.020  INFO 55726 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /hardcore_burnell: Waiting for 120 seconds for URL: http://localhost:32871/

It seems that @DynamicPropertySource is called only once when several classes are extending a base class with this annotation.

HTH

Comment From: sbrannen

What happens if you annotate your base test class with @DirtiesContext?

Comment From: sbrannen

What happens if you annotate your base test class with @DirtiesContext?

Doing that will cause a new ApplicationContext to be loaded for each subclass, which is really what you want since the Spring environment properties are changing for each subclass (i.e., the port in the example).

I have a local fix in place that tracks the testClass in DynamicPropertiesContextCustomizer in addition to the methods. The actual fix is then an equality check added to equals() like the following.

public boolean equals(Object obj) {
    if (this == obj) {
        return true;
    }
    if (obj == null || getClass() != obj.getClass()) {
        return false;
    }
    DynamicPropertiesContextCustomizer that = (DynamicPropertiesContextCustomizer) obj;
    return this.testClass.equals(that.testClass) && this.methods.equals(that.methods);
}

This change forces a new ApplicationContext to be created for subclasses of a test class that uses a @DynamicPropertySource. I think this makes sense in general.

@philwebb, what are your thoughts on the matter?

track the testClass as above?
stop comparing the methods in equals() (and just compare the testClass)?
any other options?

Comment From: aheritier

Hi @sbrannen

Thanks a lot for looking at this issue.

I tested and confirm that using @DirtiesContext in the parent class is solving the issue.

Not sure if the behaviour of @DynamicPropertySource should be changed/fixed maybe there are some legit use cases where it's not wanted/needed to reset the context.

Cheers

Comment From: sbrannen

Thanks a lot for looking at this issue.

de rien (you're welcome)

I tested and confirm that using @DirtiesContext in the parent class is solving the issue.

Thanks for confirming.

Not sure if the behaviour of @DynamicPropertySource should be changed/fixed maybe there are some legit use cases where it's not wanted/needed to reset the context.

Yeah, I have also been pondering that. In the end it depends on whether the dynamic properties change for every lookup. In your case, they do: you get a different port number every time. But in other scenarios, the dynamic properties might not be that dynamic: they might rather just be unknown but unchanging for the current environment.

With that in mind, I'm considering converting this into a documentation issue.

@philwebb, thoughts?

Comment From: philwebb

Is there a sample with some code I can look at? I'm not sure I've totally got a handle on the setup.

Comment From: aheritier

@philwebb no but I can build one tomorrow it shouldn't be hard

Comment From: aheritier

Hi @philwebb

I just created https://github.com/aheritier/spring-framework-25850 to demo the issue You need to have docker available on your host. By default the tests will pass because they are too quick (doing nothing) to have the 1st container stopped but you will see that in tests logs:

2020-10-07 09:40:12.061  INFO 43732 --- [           main] c.e.DynamicPropertySource.ParentTest     : SPRING-25850: Configuring SPRING BOOT to use ElasticSearch on: localhost:32799
2020-10-07 09:40:13.395  INFO 43732 --- [           main] c.e.DynamicPropertySource.ParentTest     : SPRING-25850: TestContainers deployed ElasticSearch on: localhost:32799
2020-10-07 09:40:22.209  INFO 43732 --- [           main] c.e.DynamicPropertySource.ParentTest     : SPRING-25850: TestContainers deployed ElasticSearch on: localhost:32801
2020-10-07 09:40:31.358  INFO 43732 --- [           main] c.e.DynamicPropertySource.ParentTest     : SPRING-25850: TestContainers deployed ElasticSearch on: localhost:32803
2020-10-07 09:40:40.504  INFO 43732 --- [           main] c.e.DynamicPropertySource.ParentTest     : SPRING-25850: TestContainers deployed ElasticSearch on: localhost:32805

The configuration of the property is done only one time while each test should use a different container

Adding @DirtiesContext In the parent class solves the issue and you will see this:

2020-10-07 09:50:17.759  INFO 50753 --- [           main] c.e.DynamicPropertySource.ParentTest     : SPRING-25850: Configuring SPRING BOOT to use ElasticSearch on: localhost:32808
2020-10-07 09:50:18.943  INFO 50753 --- [           main] c.e.DynamicPropertySource.ParentTest     : SPRING-25850: TestContainers deployed ElasticSearch on: localhost:32808
2020-10-07 09:50:27.721  INFO 50753 --- [           main] c.e.DynamicPropertySource.ParentTest     : SPRING-25850: Configuring SPRING BOOT to use ElasticSearch on: localhost:32810
2020-10-07 09:50:27.899  INFO 50753 --- [           main] c.e.DynamicPropertySource.ParentTest     : SPRING-25850: TestContainers deployed ElasticSearch on: localhost:32810
2020-10-07 09:50:35.821  INFO 50753 --- [           main] c.e.DynamicPropertySource.ParentTest     : SPRING-25850: Configuring SPRING BOOT to use ElasticSearch on: localhost:32812
2020-10-07 09:50:35.986  INFO 50753 --- [           main] c.e.DynamicPropertySource.ParentTest     : SPRING-25850: TestContainers deployed ElasticSearch on: localhost:32812
2020-10-07 09:50:43.941  INFO 50753 --- [           main] c.e.DynamicPropertySource.ParentTest     : SPRING-25850: Configuring SPRING BOOT to use ElasticSearch on: localhost:32814
2020-10-07 09:50:44.096  INFO 50753 --- [           main] c.e.DynamicPropertySource.ParentTest     : SPRING-25850: TestContainers deployed ElasticSearch on: localhost:32814

Thus the question is about the behaviour of @DynamicPropertySource. Should it stay like this and better documented (maybe it is but I didn't find) ? Should it be configurable ? Should you change the default behaviour (not sure - and there are perhaps some legit use cases where the current behaviour is the wanted one).

HTH

Cheers

Comment From: sbrannen

Thanks for providing the sample application, @aheritier.

I am repurposing this issue to add a note to the documentation that @DirtiesContext may be necessary for some @DynamicPropertySource use cases.

Spring Document @DynamicPropertySource use cases that require @DirtiesContext

@Container and @DynamicPropertySource in a parent class

@Container in a parent class and @DynamicPropertySource in every test class