Redis avoiding memmove for faster processing of redis commands during pipeline operations

While using redis with pipeline option, multiple commands are queued up in query buffer where it is held for processing. memmove() is called from sdsrange() function which moves the next command in the buffer to the beginning of the querybuf, after processing each command that is queued up.

This memmove() consumes ~10% of the time with higher P values.

Usage of memmove() can be avoided if index of next command to be processed is stored in a variable (qbufpos). The index variable is updated after processing of every command and reset when the query buffer is refilled with the new pipeline request. The optimization seen with this change is significant.

With the below command on Intel machine, we see around 10-15% improvement. redis-benchmark -h 192.168.3.100 -p 10001 -c 50 -n 200000 -d 10 -k 1 -r 10000 -P 100 get

Redis version tested : 4.0.2 OS : (Linux) Ubuntu/ Redhat

Other architecture where memory operations are slow, will see significant improvement in redis benchmarks.

Attaching the code fix herewith: 0001-Redis-optimization-Avoid-memmove-after-processing-ev.zip

Thanks, Ashutosh Parida Qualcomm Datacenter Technologies

Comment From: antirez

Hello and thank you for submitting this patch. 10-15% during pipelining is a remarkable result, so the patch will be reviewed for inclusion even at the cost of adding some complexity (I've yet to look at it). Thanks! News soon.

Comment From: aparida1

Hi Salvatore, Thanks for the response. Just wondering if you got a chance to take a look into this.

Thanks, Ashutosh

Comment From: antirez

Sorry yet not but will do in the next couple of days. Thanks and sorry for the delay!

Comment From: aparida1

Pinging to see if you got any chance to look into this and also if you need any help from my side. Looking forward for this patch.

Comment From: aparida1

@junhe77 Incase you are interested to have a look at this improvement and pitch for this.

Comment From: filipecosta90

happily reviewing the patch and coming back with up to date numbers in the next days.

Comment From: hpatro

@filipecosta90 Did you ever get to this?