MGET key [key ...] <- what's the limit to number of keys? pls put this into the doc thx
Comment From: oranagra
the only limit i can think of is the mbulk count limit of the protocol that's limited to 1,000,000 arguments. so in theory MGET would be limited to 999,999 keys. but chances are that you'll run out of memory before hitting this limit. i.e. redis would copy the values into the client reply buffers, and then either disconnect the client due to client buffer limits, or (with default configuration), it could cause the eviction mechanism to evict all the keys in the database. another possible outcome (if there are not client output buffer limits) is an OOM kill by the OS, or redis panics when it's unable to allocate memory.
so... do you still think the 999,999 limit should be documented?
Comment From: hiqsociety
of course it should be documented and explained.
Comment From: yoav-steinberg
Note that it's entirely possible to pass hundreds of thousands of arguments to a redis command. This is a meaningful use case and in many cases wouldn't cause OOM, hit buffer limits or key eviction. For example, imagine you have 500,000 keys storing integers called k0 .. k499999 and your redis is configure with 1GB of RAM. It might make sense for you to do:
MGET k0 k1 k2 .. k499999
The memory overhead of the command and its response buffers won't even exceed 10MB.
Comment From: yoav-steinberg
@itamarhaber Any idea where would be a good place in the docs to put this in?
Comment From: oranagra
I'm not sure it should be documented.. it's not an inherent limitation, it may soon be gone. But maybe @itamarhaber can remember / locate any complaints from the past that this limit bothers people with actual use-cases?
Comment From: hiqsociety
@yoav-steinberg provided very good explanation. obviously there's a reason people ask this question... for those serious in big data. just wait til data becomes smaller in future
Comment From: oranagra
yes, Yoav is right, there are cases where this limit will hit first before we run out of memory. but note that it may still be unwise to use such long queries, in some databases it can induce massive eviction, or other issues.
but i still don't think we wanna document this, instead we wanna fix this and remove the limitation (#9023). when we'll do that we'll probably mention it in the release notes, but i don't think there's a place to mention limitations of old versions in the docs.
@hiqsociety IIRC you wrote somewhere that you're writing some accessor, and that it should be compatible with other redis commands compatible dbs (each may have a different set of limitations, or implications / side effects of using extremely large queries), in that case the limit in one implementation may not matter to you. I think users may want to avoid such extreme queries, and break them down to smaller batches.
Comment From: yoav-steinberg
I agree not documenting it for now. If anyone wants to issue a PR for #9023 it'll be great!