RedisReactiveHealthIndicator health is down with cluster mode.
Check different property if ReactiveRedisClusterConnection
private Health up(Health.Builder builder, Properties info) {
return builder.up()
.withDetail(RedisHealthIndicator.VERSION, info.getProperty(RedisHealthIndicator.REDIS_VERSION)).build();
}
Comment From: wilkinsona
@mp911de What's the equivalent of org.springframework.data.redis.connection.RedisClusterCommands.clusterGetClusterInfo()
in the reactive API please?
Comment From: mp911de
Redis Cluster commands (CLUSTER INFO
) are not exposed through a reactive API. I filed DATAREDIS-1150 to provide a reactive variant of RedisClusterCommands
.
Comment From: wilkinsona
Thanks, @mp911de. Will that sneak into a Neumann SR or be a new feature in Ockham? If it's the latter, is there anything that we can do in the meantime?
Comment From: mp911de
I need to discuss this with the team. Introducing new API in service releases is always subject to cause a certain degree of confusion. There's currently no workaround.
Comment From: marcus-bcl
I've just hit this issue since upgrading to 2.3.0.RELEASE and have had to temporarily disable the Redis health endpoint, which isn't ideal (management.health.redis.enabled=false
).
As a workaround until the change is implemented in spring-data-redis, would falling back to the non-reactive RedisHealthIndicator work? Something like:
@SpringBootApplication(exclude = RedisReactiveHealthContributorAutoConfiguration.class)
Appreciate any info you can provide - thanks!
Comment From: wilkinsona
With thanks to @mp911de and the Data team, they've agreed to add the missing cluster-related APIs in a Spring Data Neumann service release. I believe it's currently slated for inclusion in Neumann-SR1 so, all being well, we should be able to fix this in Boot 2.3.1.
Comment From: andreacomo
Waiting for Boot version 2.3.1, my workaround is:
- disable default health check (
management.health.redis.enabled=false
) - Reimplement
RedisReactiveHealthIndicator
with a fixed version ofup
method like this:
private Health up(Health.Builder builder, Properties info, ReactiveRedisConnection connection) {
if (connection instanceof ReactiveRedisClusterConnection) {
List<Map<String, String>> details = getDetails(info);
if (details.isEmpty()) {
return builder.outOfService()
.build();
} else {
return builder.up()
.withDetail("nodes", details)
.build();
}
} else {
return builder.up()
.withDetail("version", info.getProperty("redis_version"))
.build();
}
}
private List<Map<String, String>> getDetails(Properties info) {
return info.keySet().stream()
.map(String.class::cast)
.map(k -> k.substring(0, k.lastIndexOf(".")))
.distinct()
.sorted()
.map(node -> Map.of(
"node", node,
"redis_version", info.getProperty(node + ".redis_version"),
"role", info.getProperty(node + ".role"),
"uptime_in_days", info.getProperty(node + ".uptime_in_days")
)
)
.collect(Collectors.toList());
}
Now my health check is UP. Hope this can help
Comment From: christopher-nash
We've experienced this same thing going from 2.2.7
to 2.2.8
.
Comment From: wilkinsona
@christopher-nash As far as I can tell, nothing has changed in this area between 2.2.7 and 2.2.8. If something was working for you in 2.2.7 and stopped working in 2.2.8, I don't think a fix for this issue will help. If you can reproduce the problem that you saw, please open a new issue with a minimal sample that works with 2.2.7 and fails with 2.2.8.
Comment From: wilkinsona
This is no longer blocked as the changes have been merged into Spring Data Neumann and Ockham (2020.0.0).
Comment From: edwardsre
@wilkinsona It looks like the properties returned in the cluster info in 2.2.8 are prefixed with the cluster node IP and port (e.g. 127.0.0.1:7001.redis_version
) so getting the redis-version
property returns null
. This is failing on the same line indicated in this issue description.
In 2.2.7.RELEASE, the properties do not have the ip and port prefix.
Comment From: wilkinsona
I can't explain that. The only Redis-related change in 2.2.8 was an upgrade to Spring Data Moore-SR8. As I said above, if you can reproduce the problem that you saw, please open a new issue with a minimal sample that works with 2.2.7 and fails with 2.2.8.
Comment From: mp911de
Spring Data Redis returns in version 2.2.8 a different connection type which reports Redis stats prefixed with host and port. See #22061 for further reference.
Comment From: scottfrederick
After some discussion, we've decided that we should not change the health response in a Spring Boot 2.3 patch release, but only change it in the 2.4 release. Boot 2.3 and earlier will continue to provide the version
field in the reactive Redis health response in both clustered and non-clustered configurations. In 2.4 the reactive and non-reactive health responses are brought into alignment, with cluster details provided when Redis is running in a clustered configuration.