When one creates its own WebServerFactory bean, the expectation is that Spring Boot backs-off completely and give you full control for configuring the chosen embedded server. This isn't the case as customizers are applied regardless (and most of the auto-configuration job is done there).
We should improve the reference guide to describe that explicitly. See #24574 for an example of exposing the factory instead of a customizer and seeing their changes overwritten by auto-configuration.