Redis redis-benchmark results differently on different kernel

Hello, everyone: When I use redis-benchmark to test redis performance on my CentOS machine, it's interesting that I got two quite different results before and after upgrading linux kernel. Actually, it seems that I have got a better performance on the older linux kernel. Why is that?

Here is my test environment and test result:

Environment:

• Alicloud ECS Centos 7.4
• redis-5.0.7

install redis: make && make test run redis-server: ./redis-5.0.7/src/redis-server ./redis-5.0.7/redis.conf run redis-benchmark: ./redis-5.0.7/src/redis-benchmark

Test result:

Before upgrading linux kernel, the version of kernel is 3.10.0-693.el7.x86_64, and here is my test result:

[root@izm5edm84ihddad1da94chz ~]# tiange/redis-5.0.7/src/redis-benchmark
====== PING_INLINE ======
  100000 requests completed in 0.91 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

99.99% <= 1 milliseconds
100.00% <= 1 milliseconds
110253.59 requests per second

====== PING_BULK ======
  100000 requests completed in 0.93 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

100.00% <= 0 milliseconds
107874.87 requests per second

====== SET ======
  100000 requests completed in 0.90 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

100.00% <= 0 milliseconds
111482.72 requests per second

====== GET ======
  100000 requests completed in 0.91 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

100.00% <= 0 milliseconds
109409.20 requests per second

====== INCR ======
  100000 requests completed in 0.90 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

100.00% <= 0 milliseconds
111607.14 requests per second

====== LPUSH ======
  100000 requests completed in 0.89 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

100.00% <= 0 milliseconds
112866.82 requests per second

====== RPUSH ======
  100000 requests completed in 0.90 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

99.94% <= 1 milliseconds
99.95% <= 4 milliseconds
100.00% <= 5 milliseconds
100.00% <= 5 milliseconds
111607.14 requests per second

====== LPOP ======
  100000 requests completed in 0.90 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

100.00% <= 1 milliseconds
111607.14 requests per second

====== RPOP ======
  100000 requests completed in 0.89 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

100.00% <= 0 milliseconds
111856.82 requests per second

====== SADD ======
  100000 requests completed in 0.90 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

99.98% <= 1 milliseconds
100.00% <= 1 milliseconds
110987.79 requests per second

====== HSET ======
  100000 requests completed in 0.87 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

100.00% <= 0 milliseconds
114942.53 requests per second

====== SPOP ======
  100000 requests completed in 0.90 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

100.00% <= 0 milliseconds
110619.47 requests per second

====== LPUSH (needed to benchmark LRANGE) ======
  100000 requests completed in 0.88 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

100.00% <= 0 milliseconds
113378.68 requests per second

====== LRANGE_100 (first 100 elements) ======
  100000 requests completed in 2.41 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

98.62% <= 1 milliseconds
100.00% <= 1 milliseconds
41545.49 requests per second

====== LRANGE_300 (first 300 elements) ======
  100000 requests completed in 6.01 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

1.39% <= 1 milliseconds
95.38% <= 2 milliseconds
99.39% <= 3 milliseconds
99.95% <= 4 milliseconds
99.97% <= 5 milliseconds
99.98% <= 6 milliseconds
99.99% <= 7 milliseconds
100.00% <= 7 milliseconds
16644.47 requests per second

====== LRANGE_500 (first 450 elements) ======
  100000 requests completed in 8.20 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

0.44% <= 1 milliseconds
27.83% <= 2 milliseconds
96.09% <= 3 milliseconds
99.15% <= 4 milliseconds
99.82% <= 5 milliseconds
99.94% <= 6 milliseconds
99.98% <= 7 milliseconds
100.00% <= 7 milliseconds
12195.12 requests per second

====== LRANGE_600 (first 600 elements) ======
  100000 requests completed in 10.34 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

0.18% <= 1 milliseconds
2.59% <= 2 milliseconds
89.83% <= 3 milliseconds
96.70% <= 4 milliseconds
98.99% <= 5 milliseconds
99.71% <= 6 milliseconds
99.96% <= 7 milliseconds
99.98% <= 8 milliseconds
99.99% <= 9 milliseconds
100.00% <= 9 milliseconds
9670.25 requests per second

====== MSET (10 keys) ======
  100000 requests completed in 1.02 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

99.95% <= 1 milliseconds
99.95% <= 4 milliseconds
100.00% <= 4 milliseconds
97847.36 requests per second

And after upgrading linux kernel, the version of kernel changes to 3.10.0-1062.12.1.el7.x86_64, and here is my test result：

[root@izm5edm84ihddad1da94chz ~]# tiange/redis-5.0.7/src/redis-benchmark
====== PING_INLINE ======
  100000 requests completed in 2.49 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

98.54% <= 1 milliseconds
99.95% <= 2 milliseconds
99.97% <= 3 milliseconds
100.00% <= 4 milliseconds
100.00% <= 4 milliseconds
40176.78 requests per second

====== PING_BULK ======
  100000 requests completed in 2.46 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

98.51% <= 1 milliseconds
100.00% <= 1 milliseconds
40600.89 requests per second

====== SET ======
  100000 requests completed in 2.48 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

97.29% <= 1 milliseconds
99.90% <= 2 milliseconds
99.93% <= 3 milliseconds
99.99% <= 4 milliseconds
100.00% <= 4 milliseconds
40322.58 requests per second

====== GET ======
  100000 requests completed in 2.49 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

98.38% <= 1 milliseconds
100.00% <= 1 milliseconds
40176.78 requests per second

====== INCR ======
  100000 requests completed in 2.46 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

98.41% <= 1 milliseconds
99.95% <= 4 milliseconds
99.98% <= 5 milliseconds
100.00% <= 5 milliseconds
40617.39 requests per second

====== LPUSH ======
  100000 requests completed in 2.48 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

98.13% <= 1 milliseconds
100.00% <= 1 milliseconds
40241.45 requests per second

====== RPUSH ======
  100000 requests completed in 2.48 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

98.28% <= 1 milliseconds
100.00% <= 1 milliseconds
40387.72 requests per second

====== LPOP ======
  100000 requests completed in 2.50 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

97.75% <= 1 milliseconds
99.95% <= 4 milliseconds
99.98% <= 5 milliseconds
100.00% <= 5 milliseconds
40064.10 requests per second

====== RPOP ======
  100000 requests completed in 2.49 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

97.89% <= 1 milliseconds
100.00% <= 1 milliseconds
40160.64 requests per second

====== SADD ======
  100000 requests completed in 2.48 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

98.34% <= 1 milliseconds
99.95% <= 4 milliseconds
99.98% <= 5 milliseconds
100.00% <= 5 milliseconds
40371.42 requests per second

====== HSET ======
  100000 requests completed in 2.47 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

97.80% <= 1 milliseconds
99.95% <= 4 milliseconds
99.98% <= 5 milliseconds
100.00% <= 5 milliseconds
40485.83 requests per second

====== SPOP ======
  100000 requests completed in 2.46 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

98.24% <= 1 milliseconds
100.00% <= 1 milliseconds
40584.41 requests per second

====== LPUSH (needed to benchmark LRANGE) ======
  100000 requests completed in 2.49 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

97.78% <= 1 milliseconds
99.95% <= 2 milliseconds
99.95% <= 4 milliseconds
99.98% <= 5 milliseconds
100.00% <= 5 milliseconds
40128.41 requests per second

====== LRANGE_100 (first 100 elements) ======
  100000 requests completed in 3.81 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

63.99% <= 1 milliseconds
98.14% <= 2 milliseconds
99.94% <= 3 milliseconds
99.97% <= 4 milliseconds
99.99% <= 5 milliseconds
100.00% <= 5 milliseconds
26239.83 requests per second

====== LRANGE_300 (first 300 elements) ======
  100000 requests completed in 7.09 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

1.00% <= 1 milliseconds
72.28% <= 2 milliseconds
95.00% <= 3 milliseconds
98.69% <= 4 milliseconds
99.92% <= 5 milliseconds
99.97% <= 6 milliseconds
99.99% <= 7 milliseconds
100.00% <= 7 milliseconds
14102.38 requests per second

====== LRANGE_500 (first 450 elements) ======
  100000 requests completed in 9.31 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

0.44% <= 1 milliseconds
10.61% <= 2 milliseconds
85.26% <= 3 milliseconds
94.81% <= 4 milliseconds
98.46% <= 5 milliseconds
99.71% <= 6 milliseconds
100.00% <= 6 milliseconds
10743.45 requests per second

====== LRANGE_600 (first 600 elements) ======
  100000 requests completed in 11.53 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

0.19% <= 1 milliseconds
2.15% <= 2 milliseconds
51.12% <= 3 milliseconds
90.92% <= 4 milliseconds
96.13% <= 5 milliseconds
98.53% <= 6 milliseconds
99.62% <= 7 milliseconds
99.96% <= 8 milliseconds
100.00% <= 9 milliseconds
100.00% <= 9 milliseconds
8669.27 requests per second

====== MSET (10 keys) ======
  100000 requests completed in 2.54 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

95.16% <= 1 milliseconds
99.89% <= 2 milliseconds
99.90% <= 3 milliseconds
99.93% <= 4 milliseconds
99.98% <= 5 milliseconds
100.00% <= 5 milliseconds
39385.59 requests per second

Comment From: SplendidSky

Finally I found it is a Linux kernel issue. It is corresponding to the patch of Spectre(CVE-2017-5753 and CVE-2017-5754). Try this: echo 0 > /sys/kernel/debug/x86/pti_enabled echo 0 > /sys/kernel/debug/x86/ibpb_enabled echo 0 > /sys/kernel/debug/x86/ibrs_enabled

Ref: https://access.redhat.com/articles/3307751 https://access.redhat.com/articles/3311301