I'm deploying my Spring Boot application into AWS. When adding an instance to an AWS load balancer, you can configure a healthcheck for the load balancer to ping to determine if an instance is healthy or not. Actuator healthcheck endpoint should work great for this. Unfortunately, AWS requires that when using an HTTP type healthcheck ping when the response returns an actual body, then Transport-Encoding set to chunked (which it is already) and a Content-Length header that is equal to or greater than Zero. The actuator healthcheck endpoint does NOT set a content-length header and it seems like this should be fairly straightforward to set. I'm going to look through the source to see if I can figure out where that addition can be made.
If this can't be added for reasons outside of my understanding, then I will have to use something other than the actuator healthcheck which seems like a big waste.
From the AWS documentation on the ELB healthcheck:
If the load balancer receives any response other than "200 OK" within the response timeout period, the instance is considered unhealthy. If the response includes a body, then your application must either set the Content-Length header to a value greater than or equal to zero, or specify Transfer-Encoding with a value set to 'chunked'.
Comment From: philwebb
The snippet of documentation that you posted seems to suggest you can use either 'chunked' Transfer-Encoding or alternatively a Content-Length. Are you actually seeing an error from AWS?
Comment From: kmandeville
Yes I am. What I am seeing is that the load balancer never thinks my instance is working properly. I see what you are saying. I definitely misread that, and from reading more about the chunked and content-length, they do seem mutually exclusive. Let me make sure I'm setting it up right.
Comment From: philwebb
This looks like it might be quite tricky to solve. Currently the HttpMessageConverter that marshals the response writes the headers before it starts streaming the body content. There is no way to know the content length without buffering the entire response first. @rstoyanchev do you have any suggestions.
I'm really surprised that a response without a content-lengh causes AWS to fail. Have you tried contacting their support (since there documentation seems to imply that it should work)?
Comment From: kmandeville
I have sent a request to AWS to find out why it's not working with Transfer-encoding coming back without Content-Length. I will see what they say
On Mon, Dec 8, 2014 at 1:02 PM, Phil Webb notifications@github.com wrote:
This looks like it might be quite tricky to solve. Currently the HttpMessageConverter that marshals the response writes the headers before it starts streaming the body content. There is no way to know the content length without buffering the entire response first. @rstoyanchev https://github.com/rstoyanchev do you have any suggestions.
I'm really surprised that a response without a content-lengh causes AWS to fail. Have you tried contacting their support (since there documentation seems to imply that it should work)?
— Reply to this email directly or view it on GitHub https://github.com/spring-projects/spring-boot/issues/2089#issuecomment-66157362 .
Comment From: celkins
@kmandeville Are you sure there is not some other configuration issue? FWIW I haven't seen any problem like what you describe (with ELB+Elastic Beanstalk):
$ curl -i http://XXXXXXXX.elasticbeanstalk.com/admin/health
HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
Date: Mon, 08 Dec 2014 21:05:01 GMT
Server: Apache-Coyote/1.1
X-Application-Context: XXXXXXXX:production
transfer-encoding: chunked
Connection: keep-alive
{"status":"UP"}
$ curl -i http://localhost:8080/admin/health
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
X-Application-Context: XXXXXXXX:production
Content-Type: application/json;charset=UTF-8
Transfer-Encoding: chunked
Date: Mon, 08 Dec 2014 21:05:03 GMT
{"status":"UP"}
Comment From: kmandeville
I am not 100% sure yet. Oh wait, I think I just thought of something... I think I know. Thanks Chris I think you helped me figure it out. It's a Security Group issue I bet!
On Mon, Dec 8, 2014 at 4:07 PM, Christopher Elkins <notifications@github.com
wrote:
@kmandeville https://github.com/kmandeville Are you sure there is not some other configuration issue? FWIW I haven't seen any problem like what you describe (with ELB+Elastic Beanstalk):
[image: screenshot 2014-12-08 12 57 26] https://cloud.githubusercontent.com/assets/362545/5347022/0cf48b20-7eda-11e4-84ab-1cac6001fbe3.png
[image: screenshot 2014-12-08 12 56 02] https://cloud.githubusercontent.com/assets/362545/5346957/9bc904ee-7ed9-11e4-801f-0985cef60a34.png
$ curl -i http://XXXXXXXX.elasticbeanstalk.com/admin/health HTTP/1.1 http://XXXXXXXX.elasticbeanstalk.com/admin/healthHTTP/1.1 200 OK Content-Type: application/json;charset=UTF-8 Date: Mon, 08 Dec 2014 21:05:01 GMT Server: Apache-Coyote/1.1 X-Application-Context: XXXXXXXX:production transfer-encoding: chunked Connection: keep-alive
{"status":"UP"}
$ curl -i http://localhost:8080/admin/health HTTP/1.1 http://localhost:8080/admin/healthHTTP/1.1 200 OK Server: Apache-Coyote/1.1 X-Application-Context: XXXXXXXX:production Content-Type: application/json;charset=UTF-8 Transfer-Encoding: chunked Date: Mon, 08 Dec 2014 21:05:03 GMT
{"status":"UP"}
— Reply to this email directly or view it on GitHub https://github.com/spring-projects/spring-boot/issues/2089#issuecomment-66187200 .
Comment From: vpiduri
Hi I am facing similar issue, can you provide your inputs.
We are running consul in OpenShift cluster. All services have been developed by Spring Boot/Cloud APIs and they have been registered successfully in consul. There is a health point exposed using SpringBoot actuator. The health point itself works just fine when try to hit using curl. But from Consul seeing a below errors frequently which causes issues in discovering the service.
Any suggestions would be great help..
2016/08/05 05:57:15 [WARN] agent: http request failed 'http://10.1.0.18:9080/health': Get http://10.1.0.18:9080/health: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[root@gtwtdlaposhv01 ~]# curl -i http://10.1.1.13:9089/health
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
X-Application-Context: spreadsheetloader-service:staging:9089
Set-Cookie: SESSION=1199bd63-c1c2-40cd-acac-218db0877123; HttpOnly
Content-Type: application/json;charset=UTF-8
Transfer-Encoding: chunked
Sometimes I do get the "status":"UP" ... sometimes I do not see this.
Comment From: wilkinsona
@vpiduri This is closed. If you believe you have found a bug, please open a new issue. If you do so, please provide as much detail as possible and, ideally, a small sample that reproduces the problem.
Comment From: saurabhdevops
@vpiduri , @wilkinsona , was there an issue open for this? I am facing a similar issue. I was wondering if There's already an issue open, I don't want to open a duplicate issue.
Comment From: wilkinsona
@saurabhdevops I don't recall seeing one
Comment From: emaayan
I am not 100% sure yet. Oh wait, I think I just thought of something... I think I know. Thanks Chris I think you helped me figure it out. It's a Security Group issue I bet!
On Mon, Dec 8, 2014 at 4:07 PM, Christopher Elkins <notifications@github.com
wrote: @kmandeville https://github.com/kmandeville Are you sure there is not some other configuration issue? FWIW I haven't seen any problem like what you describe (with ELB+Elastic Beanstalk): [image: screenshot 2014-12-08 12 57 26] https://cloud.githubusercontent.com/assets/362545/5347022/0cf48b20-7eda-11e4-84ab-1cac6001fbe3.png [image: screenshot 2014-12-08 12 56 02] https://cloud.githubusercontent.com/assets/362545/5346957/9bc904ee-7ed9-11e4-801f-0985cef60a34.png $ curl -i http://XXXXXXXX.elasticbeanstalk.com/admin/health HTTP/1.1 http://XXXXXXXX.elasticbeanstalk.com/admin/healthHTTP/1.1 200 OK Content-Type: application/json;charset=UTF-8 Date: Mon, 08 Dec 2014 21:05:01 GMT Server: Apache-Coyote/1.1 X-Application-Context: XXXXXXXX:production transfer-encoding: chunked Connection: keep-alive {"status":"UP"} $ curl -i http://localhost:8080/admin/health HTTP/1.1 http://localhost:8080/admin/healthHTTP/1.1 200 OK Server: Apache-Coyote/1.1 X-Application-Context: XXXXXXXX:production Content-Type: application/json;charset=UTF-8 Transfer-Encoding: chunked Date: Mon, 08 Dec 2014 21:05:03 GMT {"status":"UP"} — Reply to this email directly or view it on GitHub #2089 (comment) .
so what was the issue?