I'm using redis replication with sentinel in five servers. Each server runs two redis instances: redis-server and redis-sentinel. One server instance is master and the other replicas.

master.conf

port 8842
tcp-backlog 1022
tcp-keepalive 300
loglevel notice
always-show-logo no

save 900 1
save 300 10
save 60 10000

stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum no
dbfilename dump.rdb
dir redis-data/redis

requirepass "secret"

# replication settings
replicaof <masterip> <masterport>
masterauth "secret"
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync no
repl-ping-replica-period 10
repl-timeout 60
repl-disable-tcp-nodelay no
replica-priority 100
min-replicas-to-write 1
min-replicas-max-lag 20

lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
lazyfree-lazy-user-del no

io-threads 2
appendonly no
latency-monitor-threshold 0
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
stream-node-max-bytes 4096
stream-node-max-entries 100
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
dynamic-hz yes
rdb-save-incremental-fsync yes

replica.conf

port 8842
tcp-backlog 1022
tcp-keepalive 300
loglevel notice
always-show-logo no

save 900 1
save 300 10
save 60 10000

stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum no
dbfilename dump.rdb
dir redis-data/redis

requirepass "secret"

# replication settings
replicaof <master-ip> 8842
masterauth "secret"
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync no
repl-ping-replica-period 10
repl-timeout 60 
repl-disable-tcp-nodelay no
replica-priority 100
min-replicas-to-write 1
min-replicas-max-lag 20

lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
lazyfree-lazy-user-del no

io-threads 2
appendonly no
latency-monitor-threshold 0
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
stream-node-max-bytes 4096
stream-node-max-entries 100
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
dynamic-hz yes
rdb-save-incremental-fsync yes

sentinel.conf

port 37271
daemonize yes
pidfile "sentinel.pid"
logfile "sentinel.log"
dir "redis-data/sentinel"

requirepass "secret"
sentinel deny-scripts-reconfig yes
sentinel auth-pass redis-seession "secret"
sentinel monitor redis-session <master-ip> 8842 2
sentinel down-after-milliseconds redis-session 15000
sentinel failover-timeout redis-session 60000
sentinel auth-pass redis-session "secret"

Without clients connected (only replicas) the redis-server uses between 15% and 21% of CPU. Changing HZ value from 10 to 100 the CPU usage changes to 1-2%.

But the documentation says:

Redis calls an internal function to perform many background tasks, like closing connections of clients in timeout, purging expired keys that are never requested, and so forth.

Not all tasks are performed with the same frequency, but Redis checks for tasks to perform according to the specified "hz" value.

By default "hz" is set to 10. Raising the value will use more CPU when Redis is idle, but at the same time will make Redis more responsive when there are many keys expiring at the same time, and timeouts may be handled with more precision.

The range is between 1 and 500, however a value over 100 is usually not a good idea. Most users should use the default of 10 and raise this up to 100 only in environments where very low latency is required. hz 10

Note: Changing the HZ value to 1 the CPU usage is: 60-80%

Update Using only 3 servers, each one having two Redis instances (master|replica and sentinel) and HZ set to 10, now the CPU usage is 5-17%. But compared to mariadb (0.01%) it is still high.

Comment From: antirez

Hi @mvasi90, what instances are using an high CPU usage, master, replica or sentinel? An idle Redis instance should consume a very small amount of CPU, very far (much lower) from the 5% you see. More informations are needed. What version are you using and in what system? Also an INFO output could be useful.

Comment From: mvasi90

Hello @antirez, Redis instances are running inside a container. The high CPU usage is in redis master instance.

Host information Distribution: Arch Linux System: Linux server-name 5.6.15-arch1-1 #1 SMP PREEMPT Wed, 27 May 2020 23:42:26 +0000 x86_64 GNU/Linux

Container info Distribution: Arch Linux Internal network, announce-ip not needed.

Redis version Redis server v=6.0.5 sha=00000000:0 malloc=jemalloc-5.2.1 bits=64 build=4f99c66454a4aa14

Redis master info
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0

# Memory
used_memory:2132064
used_memory_human:2.03M
used_memory_rss:7749632
used_memory_rss_human:7.39M
used_memory_peak:2233928
used_memory_peak_human:2.13M
used_memory_peak_perc:95.44%
used_memory_overhead:2010898
used_memory_startup:809448
used_memory_dataset:121166
used_memory_dataset_perc:9.16%
allocator_allocated:2346480
allocator_active:2801664
allocator_resident:5971968
total_system_memory:16792723456
total_system_memory_human:15.64G
used_memory_lua:37888
used_memory_lua_human:37.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.19
allocator_frag_bytes:455184
allocator_rss_ratio:2.13
allocator_rss_bytes:3170304
rss_overhead_ratio:1.30
rss_overhead_bytes:1777664
mem_fragmentation_ratio:3.64
mem_fragmentation_bytes:5619112
mem_not_counted_for_evict:0
mem_replication_backlog:1048576
mem_clients_slaves:33972
mem_clients_normal:118902
mem_aof_buffer:0
mem_allocator:jemalloc-5.2.1
active_defrag_running:0
lazyfree_pending_objects:0

# Persistence
loading:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1593458864
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:0
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:610304
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0

# Stats
total_connections_received:29
total_commands_processed:2907913
instantaneous_ops_per_sec:6
total_net_input_bytes:150758796
total_net_output_bytes:978632997
instantaneous_input_kbps:0.38
instantaneous_output_kbps:1.34
rejected_connections:0
sync_full:3
sync_partial_ok:2
sync_partial_err:3
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:9373
evicted_keys:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:1
pubsub_patterns:0
latest_fork_usec:13186
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0

# Replication
role:master
connected_slaves:2
min_slaves_good_slaves:2
slave0:ip=<slave-ip>,port=<port>,state=online,offset=94735352,lag=0
slave1:ip=<slave-ip>,port=<port>,state=online,offset=94735352,lag=0
master_replid:<id>
master_replid2:<id>
master_repl_offset:94735498
second_repl_offset:2289337
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:93686923
repl_backlog_histlen:1048576

# CPU
used_cpu_sys:961.093915
used_cpu_user:29028.350056
used_cpu_sys_children:0.019918
used_cpu_user_children:0.012351

# Modules

# Cluster
cluster_enabled:0

# Keyspace
Redis slave info
used_memory_human:1.98M
used_memory_rss:7774208
used_memory_rss_human:7.41M
used_memory_peak:2197464
used_memory_peak_human:2.10M
used_memory_peak_perc:94.48%
used_memory_overhead:1994112
used_memory_startup:809648
used_memory_dataset:82032
used_memory_dataset_perc:6.48%
allocator_allocated:2281504
allocator_active:2723840
allocator_resident:6074368
total_system_memory:16792715264
total_system_memory_human:15.64G
used_memory_lua:37888
used_memory_lua_human:37.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.19
allocator_frag_bytes:442336
allocator_rss_ratio:2.23
allocator_rss_bytes:3350528
rss_overhead_ratio:1.28
rss_overhead_bytes:1699840
mem_fragmentation_ratio:3.82
mem_fragmentation_bytes:5740584
mem_not_counted_for_evict:0
mem_replication_backlog:1048576
mem_clients_slaves:0
mem_clients_normal:135888
mem_aof_buffer:0
mem_allocator:jemalloc-5.2.1
active_defrag_running:0
lazyfree_pending_objects:0

# Persistence
loading:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1593458864
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0

# Stats
total_connections_received:7
total_commands_processed:1027021
instantaneous_ops_per_sec:6
total_net_input_bytes:78435518
total_net_output_bytes:410508955
instantaneous_input_kbps:0.57
instantaneous_output_kbps:1.65
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:1
pubsub_patterns:0
latest_fork_usec:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0

# Replication
role:slave
master_host:<master-ip>
master_port:<master-port>
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:92828710
slave_priority:101
slave_read_only:1
connected_slaves:0
min_slaves_good_slaves:0
master_replid:<id>
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:92828710
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:91780135
repl_backlog_histlen:1048576

# CPU
used_cpu_sys:414.512690
used_cpu_user:8910.969873
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000

# Modules

# Cluster
cluster_enabled:0

# Keyspace

Redis sentinel (master side) info
# Server
redis_version:6.0.5
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:4f99c66454a4aa14
redis_mode:sentinel
os:Linux 5.6.15-arch1-1 x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:10.1.0
process_id:20
run_id:<id>
tcp_port:35189
uptime_in_seconds:439182
uptime_in_days:5
hz:10
configured_hz:10
lru_clock:16574765
executable:/home/user/redis-sentinel
config_file:/home/user/redis-config/sentinel.conf

# Clients
connected_clients:3
client_recent_max_input_buffer:2
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0

# CPU
used_cpu_sys:961.627091
used_cpu_user:681.985305
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000

# Stats
total_connections_received:7
total_commands_processed:1273665
instantaneous_ops_per_sec:2
total_net_input_bytes:74591956
total_net_output_bytes:7629395
instantaneous_input_kbps:0.11
instantaneous_output_kbps:0.01
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:12514
evicted_keys:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0

# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=redis-session,status=ok,address=<master-ip>:<master-port>,slaves=2,sentinels=3

Comment From: simplenotezy

@mvasi90 did you ever progress this? I am considering modifying the hz value