When we create a test server we don't specify an address for it to listen on and we request an ephemeral port. When we connect to the server we often use localhost. This can, I think, lead to the server listening on IPv6 ::0 but the client prefers IPv4. Normally this wouldn't cause a problem as nothing would be listening on the port in the IPv4 stack so the client would then connect to the server being tested using the IPv6 stack. However, if another process was listening to the port in the IPv4 stack, the client would connect to the wrong server. I've seen failures locally when, for example, the test accidentally connects to an HTTP server running inside IntelliJ IDEA, causing it to fail. We made some changes in this area in the past (https://github.com/spring-projects/spring-boot/commit/fedc4647e185826d512a96aa720c7df38fdcbb8e and https://github.com/spring-projects/spring-boot/commit/e973eaf2c3552c1bba431d0d3e4aef75e3ca8995). We need to do so consistently across all of our tests. Alternatively, we could configure all test tasks to run with -Djava.net.preferIPv4Stack=true but this would not take effect when running tests in an IDE that doesn't delegate to Gradle.

Comment From: wilkinsona

I've been trying to understand exactly what's happening here and it has been rather eye-opening. I've been experimenting with a modified version of our FreeMarker sample:

package smoketest.freemarker;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class SampleWebFreeMarkerApplication {

    public static void main(String[] args) {
        SpringApplication.run(SampleWebFreeMarkerApplication.class, "--application.message=no-address");
        SpringApplication.run(SampleWebFreeMarkerApplication.class, "--server.address=127.0.0.1",
                "--application.message=IPv4-address");
        SpringApplication.run(SampleWebFreeMarkerApplication.class, "--server.address=::1",
                "--application.message=IPv6-address");
    }

}

To my surprise, all three apps run successfully in parallel. The first binds to the wildcard addresses for both IPv4 and IPv6 and requests to http://localhost:8080, http://127.0.0.1:8080, and http://[::1]:8080 all result in a response containing no-address. When the second app starts, it binds to 8080 in the IPv4 stack. At this point requests to http://localhost:8080 and http://[::1]:8080 result in a response containing no-address as before, but the response to requests to http://127.0.0.1:8080 contains IPv4-address. When the third app starts, it binds to 8080 in the IPv6 stack. At this point requests to http://localhost:8080 and http://[::1]:8080 result in a response containing IPv6-address. http://127.0.0.1 continues to respond with IPv4-address.

With all three apps running, netstat shows the following:

$ netstat -n -a | grep 8080
tcp6       0      0  ::1.8080               *.*                    LISTEN     
tcp4       0      0  127.0.0.1.8080         *.*                    LISTEN     
tcp46      0      0  *.8080                 *.*                    LISTEN

Comment From: wilkinsona

IDEA appears to bind to an ephemeral port but only in the IPv4 stack. In the examples below, it's bound to 63342 and I have started a Spring Boot application with server.port=63342 and no server.address configured.

Using curl:

127.0.0.1:63342
<!doctype html><title>404 Not Found</title><h1 style="text-align: center">404 Not Found</h1><hr/><p style="text-align: center">IntelliJ IDEA 2020.1.2</p>
localhost:63342
<!DOCTYPE html>

<html lang="en">

<body>
    Date: 20-Aug-2021
    <br>
    Time: 10:53:16
    <br>
    Message: no-address
</body>

</html>
[::1]:63342
<!DOCTYPE html>

<html lang="en">

<body>
    Date: 20-Aug-2021
    <br>
    Time: 10:53:44
    <br>
    Message: no-address
</body>

</html>

From within the JVM, behaviour is slightly different with localhost being routed to the IPv4 stack:

127.0.0.1:63342
<!doctype html><title>404 Not Found</title><h1 style="text-align: center">404 Not Found</h1><hr/><p style="text-align: center">IntelliJ IDEA 2020.1.2</p>
localhost:63342
<!doctype html><title>404 Not Found</title><h1 style="text-align: center">404 Not Found</h1><hr/><p style="text-align: center">IntelliJ IDEA 2020.1.2</p>
[::1]:63342
<!DOCTYPE html>

<html lang="en">

<body>
    Date: 20-Aug-2021
    <br>
    Time: 11:06:46
    <br>
    Message: no-address
</body>

</html>

It's this behaviour with localhost in the JVM that I believe causes tests to fail occasionally. It can happen whenever there's another process listening on an ephemeral port in only the IPv4 stack. If the process has bound to the port in both stacks the problem will not occur as the Spring Boot application will not try to use the same ephemeral port.

Comment From: wilkinsona

Avoiding this potential problem is pretty ugly with @SpringBootTest configured to use a random port. To be certain that the problem will be avoided, I think the server needs to bind to a specific address and the HTTP client needs to use that same address. AFAIK, InetAddress.getLoopbackAddress().getHostAddress() is the way to get that address without making any assumptions about the host's network. It can't be used within the properties attribute of @SpringBootTest so you have to use DynamicPropertySource instead:

@DynamicPropertySource
static void serverAddress(DynamicPropertyRegistry registry) {
    registry.add("server.address", InetAddress.getLoopbackAddress()::getHostAddress);
}

You then have to apply similar configuration to the HTTP client:

```java @BeforeEach void webTestClient() { this.client = WebTestClient.bindToServer() .baseUrl("http://" + InetAddress.getLoopbackAddress().getHostAddress() + ":" + this.port) .build(); }

Comment From: sbrannen

AFAIK, InetAddress.getLoopbackAddress().getHostAddress() is the way to get that address without making any assumptions about the host's network.

@wilkinsona, do you think org.springframework.util.SocketUtils.SocketType.TCP.{...}.isPortAvailable(int) should be updated to acquire the InetAddress for localhost like that?

It currently uses InetAddress.getByName("localhost").

Comment From: wilkinsona

I don't really know yet. I need to spend some more time understanding exactly what's happening here and how specific the behaviour that I have observed is to macOS or macOS with a particular network setup.

Comment From: wilkinsona

This hasn't been such a problem recently. As such, the time it'll take to fix it is greater than the benefit it'll bring. Closing, for now at least.