First bug

127.0.0.1:28640> setbit foo 0 1
(integer) 0
127.0.0.1:28640> bitpos foo 0
(integer) 1
127.0.0.1:28640> bitpos foo 0 1
(integer) -1   # expected 8

In the docs, its mentioned that finding 0 should always assume the string is padded to infinity with 0, but the last line shows that if an out-of-bounds start is given, it assumes there are no more 0 (so -1 is returned).

Second bug

# bar is empty
> bitpos bar 0 1
(integer) 0  # should be 8

So if the value is empty, doing bitpos will return 0 if you search for zero, no matter which byte you start from (and its after the end of the "string", so its inconsistent with the first bug).

Note about start-end

On a slightly related note -- the fact that start and end expect bytes and all other operations work in bits is to say the least confusing. Also, if I know that a particular byte has a few ones in the middle, and want to search for zeros after the ones, I can't -- because I have to start at the beginning of the byte, which will find the zeros.

I know about this one
    V
000111000   < single byte
     ^
I want to search for 0 from here
^
will find this zero
      ^
instead of this one

So its not really a bug, but a big shortcoming in the API -- if it was bit based both of these problems will not exist.

Comment From: cheprasov

Hello @ichernev , I wrote a LUA script, that helps to search positions of bits by offset in bits (not bytes). if interested, please, see here: https://github.com/cheprasov/lua-scripts-for-redis/blob/master/get-bit-positions.md

Comment From: antirez

Hello @ichernev, the two bugs are clearly there, and also your final consideration is true. The API is as it looks like since all the range APIs involving strings are about bytes, since a byte after all is the unit of Redis strings. However in the case of the bit positions, this was a design mistake, and instead one should be able to specify bits.

However, while we cannot fix it and break everything, what we can do is to add a new form for the same commands, so that it is possible to specify bits. For instance:

BITPOS key 0 BITRANGE 100 200

In this case, BITRANGE is an option that can be passed to the command to modify how the arguments are parsed. We may also add BYTERANGE for symmetry, and just assume by default that the option is BYTERANGE so that there is no backward compatibility issue.

Comment From: ichernev

Hi @antirez

Adding BITRANGE could fix the 3 problems, but if BYTERANGE behaves the same as now, both problem 1 and 2 will stay (for backwards compatibility).

So I guess it makes sense to add BITRANGE, make it non-buggy :) and document these shortcomings (the bugs I listed) in the documentation so they become features :)

BITRANGE can support a single (start-only argument) and negative end for relative-to-end pointer, same as BYTERANGE and the rest of the APIs.

Comment From: antirez

@ichernev sorry I was not clear enough, I mean to fix the bugs and make part of the specification the bugfixes, breaking backward compatibility (in versions >= 4.0) without too much regrets ;-) Since I hope that

  1. It is highly unlikely that people rely on the buggy behavior (hopefully).
  2. The documentation actually documents the "new" behavior.
  3. We can add a note in the 4.0-final change log... so that everybody is advised.

Fixing the bugs, plus BITRANGE, should allow us to return to a decent setup :-) Even if not perfect, that is, the best would be just to have the bit positions, mixing bytes and bits was just my error.

Sure, negative bit positions will be supported. Instead when the end argument is missing, what to do is not exactly clear... but I guess that in that case our "end" is the last bit set in the bitmap.

Comment From: ichernev

@antirez ah, yes. That sounds even better. I also hope people don't depend on the buggy functionality, nobody else has even reported it :)

Instead of BITRANGE and BYTERANGE we can use BITID / BYTEID or BITS / BYTES so a single argument won't stay weird.

Comment From: antirez

BITS / BYTES sound good indeed :-)

Comment From: cheprasov

Hi @antirez Could you please try to add LIMIT parameter to BITPOS command. Like,

BITPOS key 0 BITS 100 200 LIMIT 25

It will help a lot for creating services based on bit operations of Redis. Thanks.

Comment From: ichernev

@cheprasov may I ask what is the LIMIT for? You already specified begin and end of the range and the command returns one result (if it could return many...).

Comment From: cheprasov

@ichernev @antirez It will be great, if BITPOS returns list of found positions of bit in range when limit is specified.

For example:

redis> BITPOS key 1 BITS 40 100
(integer) 42

redis> BITPOS key 1 BITS 40 100 LIMIT 5
1) (integer) 42
2) (integer) 44
3) (integer) 47
4) (integer) 55
5) (integer) 88

redis> BITPOS key 1 LIMIT 10
1) (integer) 1
2) (integer) 3
3) (integer) 4
...

Comment From: Almya

I would love to see this feature see the light one day