See https://issues.redhat.com/browse/UNDERTOW-1705 for details of the problem. We only see the flakiness on Windows where we're unable to delete the temporary text file that's been served by Undertow, presumably because it has leaked a file handle. I suspect that it won't be possible to close the file handle once the NullPointerException has occurred so we need to prevent the NPE from happening. We could try sleeping for a little while between receiving the response and stopping the server. Alternatively, we could try to delete the text file before stopping the server. When running on Windows, we'll only be able to delete the text file once it's been closed.