I'm fairly new to Microservices world and trying to build a project while learning new concepts. I've been struck on this issue for quite a long time and saw that it might be a possible duplicate but I've tried all the solutions from another posts. Basically I have four applications deployed to Heroku:
Config-Server Eureka-Server Front-End-Service User-Service I was trying to replicate Facebook application hence the prefix 'friendsbook' is attached to each microservice name. The UI part of application will only hit API endpoints exposed by Front-End service - It's sort of like a Zuul Service. Anyhow,
2, #3 and #4 are config clients of #1.
3 and #4 are eureka clients of #2.
3 and #4 successfully registers themselves to Eureka Server as seen in below image
https://ibb.co/9TBnS2N
The User-Service is protected by basic authentication. I'm using Feign Client to call user-service from front-end service. Everything works fine when I run the thing locally but in Heroku only I get this issue. I'm able to access the User service from postman with basic auth credentials. I've tried restTemplate as well but it throws the same error.
The log files at front-end service:
2020-12-20T18:12:24.532585+00:00 app[web.1]: 2020-12-20 18:12:24.532 INFO 4 --- [io-27853-exec-3] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
2020-12-20T18:12:24.534988+00:00 app[web.1]: 2020-12-20 18:12:24.534 INFO 4 --- [io-27853-exec-3] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
2020-12-20T18:12:24.642618+00:00 app[web.1]: 2020-12-20 18:12:24.642 INFO 4 --- [io-27853-exec-3] o.s.web.servlet.DispatcherServlet : Completed initialization in 107 ms
2020-12-20T18:12:26.712658+00:00 app[web.1]: 2020-12-20 18:12:26.712 INFO 4 --- [io-27853-exec-3] c.netflix.config.ChainedDynamicProperty : Flipping property: friendsbook-user-service.ribbon.ActiveConnectionsLimit to use NEXT property: niws.loadbalancer.availabilityFilteringRule.activeConnectionsLimit = 2147483647
2020-12-20T18:12:27.025344+00:00 app[web.1]: 2020-12-20 18:12:27.025 INFO 4 --- [io-27853-exec-3] c.n.u.concurrent.ShutdownEnabledTimer : Shutdown hook installed for: NFLoadBalancer-PingTimer-friendsbook-user-service
2020-12-20T18:12:27.043264+00:00 app[web.1]: 2020-12-20 18:12:27.025 INFO 4 --- [io-27853-exec-3] c.netflix.loadbalancer.BaseLoadBalancer : Client: friendsbook-user-service instantiated a LoadBalancer: DynamicServerListLoadBalancer:{NFLoadBalancer:name=friendsbook-user-service,current list of Servers=[],Load balancer stats=Zone stats: {},Server stats: []}ServerList:null
2020-12-20T18:12:27.152239+00:00 app[web.1]: 2020-12-20 18:12:27.151 INFO 4 --- [io-27853-exec-3] c.n.l.DynamicServerListLoadBalancer : Using serverListUpdater PollingServerListUpdater
2020-12-20T18:12:27.360183+00:00 app[web.1]: 2020-12-20 18:12:27.359 INFO 4 --- [io-27853-exec-3] c.netflix.config.ChainedDynamicProperty : Flipping property: friendsbook-user-service.ribbon.ActiveConnectionsLimit to use NEXT property: niws.loadbalancer.availabilityFilteringRule.activeConnectionsLimit = 2147483647
2020-12-20T18:12:27.376918+00:00 app[web.1]: 2020-12-20 18:12:27.375 INFO 4 --- [io-27853-exec-3] c.n.l.DynamicServerListLoadBalancer : DynamicServerListLoadBalancer for client friendsbook-user-service initialized: DynamicServerListLoadBalancer:{NFLoadBalancer:name=friendsbook-user-service,current list of Servers=[friendsbook-user-service.herokuapp.com:51003],Load balancer stats=Zone stats: {defaultzone=[Zone:defaultzone; Instance count:1; Active connections count: 0; Circuit breaker tripped count: 0; Active connections per server: 0.0;]
2020-12-20T18:12:27.376925+00:00 app[web.1]: },Server stats: [[Server:friendsbook-user-service.herokuapp.com:51003; Zone:defaultZone; Total Requests:0; Successive connection failure:0; Total blackout seconds:0; Last connection made:Thu Jan 01 00:00:00 UTC 1970; First connection made: Thu Jan 01 00:00:00 UTC 1970; Active Connections:0; total failure count in last (1000) msecs:0; average resp time:0.0; 90 percentile resp time:0.0; 95 percentile resp time:0.0; min resp time:0.0; max resp time:0.0; stddev resp time:0.0]
2020-12-20T18:12:27.376926+00:00 app[web.1]: ]}ServerList:org.springframework.cloud.netflix.ribbon.eureka.DomainExtractingServerList@4243762b
2020-12-20T18:12:27.465402+00:00 app[web.1]: 2020-12-20 18:12:27.465 INFO 4 --- [io-27853-exec-3] DeferredRepositoryInitializationListener : Triggering deferred initialization of Spring Data repositories…
2020-12-20T18:12:27.465943+00:00 app[web.1]: 2020-12-20 18:12:27.465 INFO 4 --- [io-27853-exec-3] DeferredRepositoryInitializationListener : Spring Data repositories initialized!
2020-12-20T18:12:28.161225+00:00 app[web.1]: 2020-12-20 18:12:28.160 INFO 4 --- [erListUpdater-0] c.netflix.config.ChainedDynamicProperty : Flipping property: friendsbook-user-service.ribbon.ActiveConnectionsLimit to use NEXT property: niws.loadbalancer.availabilityFilteringRule.activeConnectionsLimit = 2147483647
Feign-Client for User-Service in Front-End code
@FeignClient(
value = "https://friendsbook-user-service/",
fallback = UserServiceClientFallback.class,
configuration = UserServiceClientConfiguration.class)
public interface UserServiceClient {
@RequestMapping(method = RequestMethod.POST, value = "/user/sign-up", consumes = "application/json")
public String createUser(@RequestHeader Map<String, String> headers,User obj);
}
The config files for this feign client
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import feign.Logger;
import feign.auth.BasicAuthRequestInterceptor;
public class UserServiceClientConfiguration {
@Value("${userservice.api.username}")
public String userServiceAuthenticationUsername;
@Value("${userservice.api.password}")
public String userServiceAuthenticationPassword;
@Bean
public BasicAuthRequestInterceptor basicAuthRequestInterceptor() {
return new BasicAuthRequestInterceptor(userServiceAuthenticationUsername, userServiceAuthenticationPassword);
}
@Bean
Logger.Level feignLoggerLevel() {
return Logger.Level.FULL;
}
}
The Controller in Front end:
import java.net.SocketTimeoutException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import com.friendsbook.frontend.model.User;
import com.friendsbook.frontend.service.UserServiceClient;
@RestController
@RequestMapping("/user")
public class UserController {
@Autowired
private UserServiceClient client;
private Logger logger = LoggerFactory.getLogger(UserController.class);
@PostMapping("/sign-up")
public String createUser(@RequestBody User obj) throws SocketTimeoutException {
return this.client.createUser(obj);
}
}
The bootstrap.yml file of front-end service
spring:
application:
name: ${APP_NAME}
cloud:
config:
uri:
- ${CONFIG-SERVER-URL}// points to the config-server web url
The application.yml file of front-end service
eureka:
instance:
hostname: ${APP_NAME}.herokuapp.com
userservice:
api:
username: ${USER_SERVICE_USERNAME}// username of basic-authenticaion user of User-Service
password: ${USER_SERVICE_PASSWORD}// password of basic-authenticaion user of User-Service
the pom file of front-end service
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.3.6.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.friendsbook</groupId>
<artifactId>FriendsBook-Front-End-Service</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>FriendsBook-Front-End-Service</name>
<description>Front end of FriendsBook Microservices - will also act as a Eureka Server</description>
<properties>
<java.version>11</java.version>
<spring-cloud.version>Hoxton.SR9</spring-cloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>
<dependency>
<groupId>io.jsonwebtoken</groupId>
<artifactId>jjwt</artifactId>
<version>0.9.1</version>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.hibernate.validator</groupId>
<artifactId>hibernate-validator</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
In front-end-service Security the /sign-up end point is public
The bootstrap.yml file of User-Service
spring:
application:
name: ${APP_NAME}
cloud:
config:
uri:
- ${CONFIG-SERVER-URL}//points to the config-server
The application.yml file of User-Service
spring:
security:
user:
name: ${SECURITY-USER-NAME}// username of basic-authenticaion user
password: ${SECURITY-USER-PASSWORD}// password of basic-authenticaion user
datasource:
url: ${DATABASE-URL}
username: ${DATABASE-USERNAME}
password: ${DATABASE-USER-PASSWORD}
driver-class-name: ${DRIVER-CLASS-NAME}
eureka:
instance:
hostname: ${APP_NAME}.herokuapp.com
The pom file of User-Service
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.3.6.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.friendsbook</groupId>
<artifactId>FriendsBook-User-Service</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>FriendsBook-User-Service</name>
<description>User Service of FriendsBook Microservices</description>
<properties>
<java.version>11</java.version>
<spring-cloud.version>Hoxton.SR9</spring-cloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.hibernate.validator</groupId>
<artifactId>hibernate-validator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
In config server side both these service fetches these common properties
eureka:
client:
fetch-registry: true
register-with-eureka: true
healthcheck:
enabled: true
service-url:
defaultZone: "{cipher}2d6cfa8111baba9cadb74e838a1cbceecdbfe8eb3b356ad3485584573efe7e66ead7be72a7ca7ce71aac35b58f66431239135bfafc5520cd4de2009f3a407b5971c3d9a6943f2eb7478bab364978dec8"
non-secure-port: 80
secure-port: 443
The encrypted part is actually Eureka Server url with '\eureka' in end.
I'm also able to see the list of servers at /eureka/apps
<?xml version="1.0" encoding="UTF-8"?>
<applications>
<versions__delta>1</versions__delta>
<apps__hashcode>UP_2_</apps__hashcode>
<application>
<name>FRIENDSBOOK-FRONT-END</name>
<instance>
<instanceId>a3062f2a-223f-43a1-a5d2-6721ff6705f4.prvt.dyno.rt.heroku.com:friendsbook-front-end:12002</instanceId>
<hostName>friendsbook-front-end.herokuapp.com</hostName>
<app>FRIENDSBOOK-FRONT-END</app>
<ipAddr>172.17.229.42</ipAddr>
<status>UP</status>
<overriddenstatus>UNKNOWN</overriddenstatus>
<port enabled="true">12002</port>
<securePort enabled="false">443</securePort>
<countryId>1</countryId>
<dataCenterInfo class="com.netflix.appinfo.InstanceInfo$DefaultDataCenterInfo">
<name>MyOwn</name>
</dataCenterInfo>
<leaseInfo>
<renewalIntervalInSecs>30</renewalIntervalInSecs>
<durationInSecs>90</durationInSecs>
<registrationTimestamp>1608573161189</registrationTimestamp>
<lastRenewalTimestamp>1608573791056</lastRenewalTimestamp>
<evictionTimestamp>0</evictionTimestamp>
<serviceUpTimestamp>1608573160659</serviceUpTimestamp>
</leaseInfo>
<metadata>
<management.port>12002</management.port>
</metadata>
<homePageUrl>http://friendsbook-front-end.herokuapp.com:12002/</homePageUrl>
<statusPageUrl>http://friendsbook-front-end.herokuapp.com:12002/actuator/info</statusPageUrl>
<healthCheckUrl>http://friendsbook-front-end.herokuapp.com:12002/actuator/health</healthCheckUrl>
<vipAddress>friendsbook-front-end</vipAddress>
<secureVipAddress>friendsbook-front-end</secureVipAddress>
<isCoordinatingDiscoveryServer>false</isCoordinatingDiscoveryServer>
<lastUpdatedTimestamp>1608573161189</lastUpdatedTimestamp>
<lastDirtyTimestamp>1608573159802</lastDirtyTimestamp>
<actionType>ADDED</actionType>
</instance>
</application>
<application>
<name>FRIENDSBOOK-USER-SERVICE</name>
<instance>
<instanceId>dd335370-7a1b-414f-84ce-e5a764680425.prvt.dyno.rt.heroku.com:friendsbook-user-service:44880</instanceId>
<hostName>friendsbook-user-service.herokuapp.com</hostName>
<app>FRIENDSBOOK-USER-SERVICE</app>
<ipAddr>172.18.27.226</ipAddr>
<status>UP</status>
<overriddenstatus>UNKNOWN</overriddenstatus>
<port enabled="true">44880</port>
<securePort enabled="false">443</securePort>
<countryId>1</countryId>
<dataCenterInfo class="com.netflix.appinfo.InstanceInfo$DefaultDataCenterInfo">
<name>MyOwn</name>
</dataCenterInfo>
<leaseInfo>
<renewalIntervalInSecs>30</renewalIntervalInSecs>
<durationInSecs>90</durationInSecs>
<registrationTimestamp>1608572957018</registrationTimestamp>
<lastRenewalTimestamp>1608573796737</lastRenewalTimestamp>
<evictionTimestamp>0</evictionTimestamp>
<serviceUpTimestamp>1608572956413</serviceUpTimestamp>
</leaseInfo>
<metadata>
<management.port>44880</management.port>
</metadata>
<homePageUrl>http://friendsbook-user-service.herokuapp.com:44880/</homePageUrl>
<statusPageUrl>http://friendsbook-user-service.herokuapp.com:44880/actuator/info</statusPageUrl>
<healthCheckUrl>http://friendsbook-user-service.herokuapp.com:44880/actuator/health</healthCheckUrl>
<vipAddress>friendsbook-user-service</vipAddress>
<secureVipAddress>friendsbook-user-service</secureVipAddress>
<isCoordinatingDiscoveryServer>false</isCoordinatingDiscoveryServer>
<lastUpdatedTimestamp>1608572957018</lastUpdatedTimestamp>
<lastDirtyTimestamp>1608572955446</lastDirtyTimestamp>
<actionType>ADDED</actionType>
</instance>
</application>
</applications>
Again, everything works fine in local but not on heroku even though ribbon has something in its list of servers. Also the log file is generate only at the first request. After that it always show request is timedout.
Comment From: spencergibb
I mentioned NOT to open another issue https://github.com/spring-cloud/spring-cloud-netflix/issues/3945#issuecomment-749128166. The stack overflow issue mentioned in #3945 is sufficient.
Comment From: Vaibhav5757
Apologies again for doing so