Redis Cluster can be reconfigured at runtime and Redis clients should to pick up these changes to stay operable. Updating the topology is a specialized topic on its own as Redis Cluster does not push changes to the client but rather clients need to poll topology changes.

Automated (background) topology refresh is currently supported only by Lettuce via ClusterClientOptions and a nested ClusterTopologyRefreshOptions object. Jedis does not support background topology refreshing.

It would make sense to provide some means to configure cluster topology refresh. The way how Lettuce topology updating is configured (setting ClusterClientOptions via LettuceClientConfigurationBuilder) might contradict with LettuceClientConfigurationBuilderCustomizer.

What do you think?

See also:

  • lettuce-io/lettuce-core#955
  • xetorthio/jedis#1347

Comment From: wilkinsona

I don't think that things would necessarily contradict. We could create and set a ClusterClientOptions based on various properties before calling the customisers. They could then further tune things as they do today.

Looking at ClusterClientOptions and ClusterTopologyRefreshOptions, we could add a lot of properties here, and, if we went down that route, I wonder if a user-provided LettuceClientConfigurationBuilderCustomizer would be a better choice.

Are there sensible defaults that could be used such that we could offer a single configuration property that, when set, would cause us to configure the builder with a ClusterClientOptions instance? In other words, does it make sense to do something like this:

if (refreshCluster) {
    builder.clientOptions(ClusterClientOptions.builder().build());
}

Comment From: mp911de

FTR: LettuceClientConfigurationBuilderCustomizer is already in place.

Are there sensible defaults

Not exactly. From the reported requests, each deployment uses different configuration settings. From a configuration perspective and how refresh is used, the most common options are:

  • not enable topology refresh at all (that's what happens today)
  • enable a light-weight refresh version (ClusterTopologyRefreshOptions.builder().enableAllAdaptiveRefreshTriggers().build())
  • periodic refresh (ClusterTopologyRefreshOptions.builder().enablePeriodicRefresh(Duration).build())
  • or a combination of both (ClusterTopologyRefreshOptions.builder().enablePeriodicRefresh(Duration).enableAllAdaptiveRefreshTriggers().build()).

In any case, we require a ClusterClientOptions object for ClusterTopologyRefreshOptions.

So how about introducing the following: * Property to enable refresh (defaults to false, would result in enableAllAdaptiveRefreshTriggers() if enabled). * Property to enable periodic refresh (no default, would result in enablePeriodicRefresh(Duration) if set and if the refresh property above is enabled).

The background is that adaptive refresh results from using Redis, so Redis responses/connection events are triggers to do a topology refresh. The periodic one requires a scheduler and generates regularly load on all Redis servers even when there was no change.

Comment From: wilkinsona

Thank you, @mp911de. So, how about the following properties:

  • spring.redis.cluster.refresh.period (Duration)
  • spring.redis.cluster.refresh.all-adaptive-triggers (boolean)

We would only create and set a ClusterClientOptions with a custom ClusterTopologyRefreshOptions if all-adaptive-triggers is true or period in not null.

  • Property to enable periodic refresh (no default, would result in enablePeriodicRefresh(Duration) if set and if the refresh property above is enabled).

Why would this require another property to be set? Isn't just configuring the refresh period sufficient to allow people to configure the middle use case in your comment above:

  • periodic refresh (ClusterTopologyRefreshOptions.builder().enablePeriodicRefresh(Duration).build())

Comment From: mp911de

Let's rename the second property to spring.redis.cluster.refresh.adaptive as this makes a good story for adaptive and periodic refresh.

Why would this require another property to be set? Isn't just configuration the refresh period sufficient to allow people to configure the middle use case in your comment above:

You're right, maybe it's better to just rely on a single property.

Comment From: arganzheng

mark

Comment From: maolujun

@mp911de Using the jedis client is successful pom.xml： SpringBoot Add configuration to enable Redis Cluster topology refresh

Comment From: ldwnt

@mp911de @wilkinsona Do you any plan for including this feature in a new release? I'd really appreciate it if the topology refresh feature could be supported by spring boot, so that my RedisTemplate related code does not need to be re-written~

Comment From: wilkinsona

None at this time. We'll probably consider it again when we're planning 2.3.

Comment From: derek82511

@ldwnt I have the same needs ... I really need this feature to connect my Redis Cluster on Kubernetes.

Comment From: snicoll

I've pushed a proposal in 80884ed. There are a few things that we still need to discuss:

  • The "generic" LettuceClientOptionsBuilderCustomizer customizes a ClusterClientOptions. Failing to do that, users wouldn't have access to the additional lettuce options. Should such options be created if cluster settings are not set? Right now a ClusterClientOptions is set no matter what
  • After a bit of back and forth, I think putting those options in the spring.redis.cluster is more consistent even if we don't have support for Jedis.
  • Because we overrides the cluster options, we have to copy/paste the default initalization that the builder applies. It's not great but the current API does not offer any other option as far as I can see.

Comment From: superbool

Why not support 2.2.x ?

Comment From: snicoll

@superbool new features are not added to maintenance releases. 2.3.x is in maintenance as well in the meantime so please upgrade to that.