Intro
Currently, the information Redis provides in COMMAND is limited and incomplete, causing proxy/client writers to seek missing information in other places.
The goal is to make Redis the single source of truth when it comes to information about its commands.
Steps:
1. Some commands have no purpose without specifying a sub-command (e.g. CONFIG, CLIENT, MEMORY, etc.). We would like to treat such sub-commands as independent commands. It’ll help with the other issues and will make everything cleaner in general (for example, different loading flag for CONFIG GET vs. CONFIG SET, different ACL categories for CLIENT KILL vs. CLIENT SETNAME). PR: https://github.com/redis/redis/pull/9504
2. Redis clients (e.g. redis-py) need to know, when parsing a command, where the key arguments are (to know to which cluster node to send a command in cluster mode). Currently, COMMAND can help only if the key positions are simple (basically it’s a range of indexes, with a key-step). While that works for some commands (SET, LPUSH, etc.) it doesn’t work for commands which have a more complicated scheme (XADD, ZUNION) so clients either have a hardcoded way to handle “complex” commands or they have to COMMAND GETKEYS (round trip). PR: https://github.com/redis/redis/pull/8324
3. Proxies need to know some information about the command (e.g. PING should be sent to all shards while GET is sent only to one shard). More information in the “Command tips” section.
4. Both redis.io and help.h rely on commands.json because Redis doesn’t provide some information (command usage, first version it was released, short description, complexity, etc.)
The goal is to have the per-command JSON as the single source of truth, containing everything there is to know about a command. We will generate code (commands.json for radis.io, the command table for Redis, etc.) using these files.
Subcommands as commands
If a command is meaningless without a subcommand (CONFIG, MEMORY, and others) it may suggest that we’re actually talking about a few separate logical commands that happen to have a common theme.
These subcommands need to have a separate entry in the command table with their own unique flags and ACL categories.
A list of such commands:
CONFIG
COMMAND
MEMORY
SLOWLOG
OBJECT
SCRIPT
DEBUG
XGROUP
XINFO
CLIENT
CLUSTER
ACL
STRALGO
MODULE
LATENCY
COMMAND command
COMMAND has the following issues:
1. All the commands above, excluding COMMAND, have no meaning with a subcommand. That means we will need to represent both COMMAND and COMMAND INFO, etc. in the command table.
2. We would like to eliminate the above commands (except COMMAND) from the command table altogether (i.e. CONFIG by itself is meaningless). We can’t do that because we have to maintain backward-compatibility of COMMAND (i.e. COMMAND INFO config should work)
COMMAND’s output (with no args) will keep the old format, with entries to all commands for backward compatibility but we will try to deprecate it.
We will add a new command COMMAND LIST, that’ll return the list of commands in a new format, which will include all the features described in this document.
The idea is that each such command will have an array of sub-commands. Once the server finds a hit on argv[0] as the command name it needs to check whether this command has subcommands and try to match argv[1]. It means that we are no longer talking about a command table, we’re talking about a command tree.
Key positions
In order to easily find the positions of keys in a given array of args we introduce keys specs. There are two logical steps of key specs:
1. start_search: Given an array of args, indicate where we should start searching for keys
2. find_keys: Given the output of start_search and an array of args, indicate all possible indices of keys.
start_search step specs
index: specify an index explicitlykeyword: specify a string to match inargv. We should start searching for keys just after the keyword appears. Another property for this spec is an index from which to start the keyword search (can be negative, which means to search from the end)
Examples:
- SET has start_search of type index with value 1
- XREAD has start_search of type keyword with value [“STREAMS”,1]
- MIGRATE has start_search of type keyword with value [“KEYS”,-2]
find_keys step specs
range: specify[count, step, limit].count: number of expected keys. indicating till the last argument, -2 one before the laststep: how many args should we skip after finding a key, in order to find the next onelimit: if count is -1, we use limit to stop the search by a factor. 0 and 1 mean no limit. 2 means ½ of the remaining args, 3 means ⅓, and so on.- “keynum”: specify
[keynum_index, first_key_index, step]. Note:keynum_indexis relative to the return of thestart_searchspec.first_key_indexis relative tokeynum_index.
Examples:
- SET has range of [1,1,0]
- MSET has range of [-1,2,0]
- XREAD has range of [-1,1,2]
- ZUNION has start_search of type index with value 1 and find_keys of type keynum with value [0,1,1]
- AI.DAGRUN has start_search of type keyword with value [“LOAD“,1] and find_keys of type keynum with value [0,1,1] (see https://oss.redislabs.com/redisai/master/commands/#aidagrun)
Note: this solution is not perfect as the module writers can come up with anything, but at least we will be able to find the key args of the vast majority of commands.
If one of the above specs can’t describe the key positions, the module writer can always fall back to the getkeys-api option.
Some keys cannot be found easily (KEYS in MIGRATE: Imagine the argument for AUTH is the string “KEYS” - we will start searching in the wrong index). The guarantee is that the specs may be incomplete (incomplete will be specified in the spec to denote that) but we never report false information.
We will expose these key specs in the COMMAND command so that clients can learn, on startup, where the keys are for all commands instead of holding hardcoded tables or use COMMAND GETKEYS in runtime.
Details: https://github.com/redis/redis/issues/7297#issuecomment-714528935 (possibly outdated). PR: https://github.com/redis/redis/pull/8324
Needless to say, when a module registers a command it should provide all that info as well, old modules that don't will only expose a basic index and range.
Command tips
The redisCommand structure should have a list of arbitrary “tips” (strings). Unlike the existing command flags (saved as a bitfield), which are internal by nature (tells Redis how to handle commands) the command tips are “external”: They are used by proxies and clients to learn how to handle the command (for example, to which shard to send the command, among other stuff).
There will be a predefined list of common tips but we will have to support arbitrary tips as well.
We will need to expose a module API for adding tips to commands.
Needless to say that modules can add commands with either similar tips as built-in commands or even arbitrary ones.
Examples:
- requesting_policy: to which shard(s) do we send the command?
Example: GET goes to a single shard (depends on the slot of the key) while PING goes to all shards.
- replying_policy: what do we do with all the replies we got from the shards? (in case the requsting_policy is to more than one shard).
Example: For CONFIG SET the proxy needs to make sure all shards returned OK. For DBSIZE the proxy needs to aggregate the output from all shards and synthesize a reply of its own.
Commands' JSON
We want Redis to have all necessary command information, so we need to save all information currently exposed in redis.io's commands.json:
- command structure, including all args and their types, what’s an enum, what’s optional, etc.
- complexity information
- short description
- which version introduced the command
- datatype (could be “general”)
We want Redis to have the ability to generate the old commands.json, making it the only source of truth regarding command information
We will have a JSON file per command
At the top level, .json contains a JSON object (hashmap), mapping command fields to their values.
Possible fields:
- summary: string describing the command's functionality
- complexity: string describing the command's performance complexity
- since: version-string with the first version number supporting the command
- group: string with the name of the command group the command belongs to
- arity: number of args this command has (where negative values mean “at least”)
- return_summary: string representing the return type(s)
- return_types: map of resp_version -> list(return-type-string)
- command_flags: list of strings
- acl_categories: list of strings
- history: string representing behavior changes in the history of the command
- key_specs: a list of objects (two keys each, start_search, find_keys - each of them is an object)
- arguments: list of argument specification dictionaries (if undefined, command takes no arguments)
Each argument dictionary can have the following fields:
- name: String. The name of the argument.
- description: String. Short description of the argument (optional)
- type: String. The type of argument. Possible values:
- "string": a string-valued argument
- "integer": an integer-valued argument
- "double": a floating-point argument
- "key": a string-valued argument representing a key in the datastore
- "pattern": a string representing a glob-style pattern
- "unix_time": integer-valued argument is a Unix timestamp value in seconds
- "oneof": multiple options that mutually exclude each other. in this case the field "value" is a list of arguments
- "block": not an individual argument, but a block of multiple arguments. in this case, the field "value" is a list of arguments
- value: String or List. Either the name to display or a list of dictionaries (each is an argument, so arguments can be nested)
- token: String. Name of the preceding token if exists (optional)
- optional: Boolean. True if this argument is optional. (optional)
- multiple: Boolean. True if this argument can be repeated multiple times. (optional)
- since: String. The first version introduced this argument. (optional)
Examples:
{
"SET": {
"summary": "Set the string value of a key",
"complexity": "O(1)",
"since": "1.0.0",
"group": "string",
"arity": -3,
"return_summary": "Simple string reply: OK if SET was executed correctly."
"Null reply: (nil) if the SET operation was not performed because the user specified the NX or XX option but the condition was not met."
"If the command is issued with the GET option, the above does not apply. It will instead reply as follows, regardless if the SET was actually performed:"
"Bulk string reply: the old string value stored at key."
"Null reply: (nil) if the key did not exist.",
"return_types": {
"2": ["+OK", "<bulk-string>", "<null-bulk-string>"],
"3": ["+OK", "<bulk-string>", "<null>"],
},
"command_flags": [
"write",
"use-memory",
],
"acl_categories": [
"string",
],
"key_specs": [
{
"start_search": {
"index": {
"pos": 1,
}
},
"find_keys": {
"range": {
"count": 1,
"step": 1,
"limit": 0,
}
},
"flags": [
"write",
],
}
],
"history": ">= 7.0: Allowed the NX and GET options to be used together.",
"arguments": [
{
"name": "key",
"type": "key",
"value": "key",
},
{
"name": "value",
"type": "string",
"value": "value",
},
{
"name": "expire",
"optional": true,
"type": "oneof",
"value": [
{
"name": "ex",
"since": "2.6.12",
"type": "integer",
"token": "EX",
"value": "seconds",
},
{
"name": "px",
"since": "2.6.12",
"type": "integer",
"token": "PX",
"value": "milliseconds",
},
{
"name": "keepttl",
"since": "6.0.0",
"type": null,
"token": "KEEPTTL",
"value": null,
},
],
},
{
"name": "existence",
"optional": true,
"type": "oneof",
"value": [
{
"name": "nx",
"type": null,
"token": "NX",
"value": null,
},
{
"name": "xx",
"type": null,
"token": "XX",
"value": null,
},
],
},
{
"name": "get",
"since": "6.2.0",
"optional": true,
"type": null,
"token": "GET",
"value": null,
},
],
}
}
{
"XADD": {
"summary": "Appends a new entry to a stream",
"complexity": "O(1) when adding a new entry, O(N) when trimming where N being the number of entires evicted.",
"since": "5.0.0",
"group": "stream",
"arity": -5,
"return_summary": "Bulk string reply, specifically:"
"The command returns the ID of the added entry. The ID is the one auto-generated if * is passed as ID argument, otherwise the command just returns the same ID specified by the user during insertion."
"The command returns a Null reply when used with the NOMKSTREAM option and the key doesn't exist.",
"return_types": {
"2": ["<bulk-string>", "<null-bulk-string>"],
"3": ["<bulk-string>", "<null>"],
},
"command_flags": [
"write",
"use-memory",
"fast",
"random",
],
"acl_categories": [
"stream",
],
"key_specs": [
{
"start_search": {
"index": {
"pos": 1,
}
},
"find_keys": {
"range": {
"count": 1,
"step": 1,
"limit": 0,
}
},
"flags": [
"write",
],
}
],
"arguments": [
{
"name": "key",
"type": "key",
"value": "key",
},
{
"name": "trimming",
"optional": true,
"type": "block",
"value": [
{
"name": "strategy",
"optional": false,
"type": "oneof",
"value": [
{
"name": "maxlen",
"type": null,
"token": "MAXLEN",
"value": null,
},
{
"name": "minid",
"since": "6.2.0",
"type": null,
"token": "MINID",
"value": null,
},
],
},
{
"name": "operator",
"optional": true,
"type": "oneof",
"value": [
{
"name": "exact",
"type": null,
"token": "=",
"value": null,
},
{
"name": "inexact",
"type": null,
"token": "~",
"value": null,
},
],
},
{
"name": "threshold",
"type": "integer",
"value": "threshold",
},
{
"name": "limit",
"optional": true,
"since": "6.2.0",
"type": "integer",
"token": "LIMIT",
"value": "limit",
},
],
},
{
"name": "nomakestream",
"optional": true,
"since": "6.2.0",
"type": null,
"token": "NOMKSTREAM",
"value": null,
},
{
"name": "id",
"type": "oneof",
"value": [
{
"name": "auto",
"type": null,
"token": "*",
"value": null,
},
{
"name": "specific",
"type": null,
"token": "ID",
"value": null,
},
],
},
{
"name": "fieldandvalues",
"type": "block",
"multiple": true,
"value": [
{
"name": "field",
"type": "string",
"value": "field",
},
{
"name": "value",
"type": "string",
"value": "value",
},
],
},
],
}
}
Updates
21.8.12: command tips should be configurable and not hardcoded: user can provide tips.json on startup. that way, every user can provide its own tips. it should support module commands as well (means we need to parse tips.json only after all modules were loaded). what about loading modules in runtime?
Comment From: JimB123
The goal is to have the per-command JSON as the single source of truth, containing everything there is to know about a command. We will generate code (commands.json for radis.io, the command table for Redis, etc.) using these files.
So, if I understand, this means that a single JSON document will be used to:
* Generate the documentation at https://redis.io/commands/
* Would the entire command document be generated from the JSON? This implies that a large quantity of text documentation would be included in the JSON.
* Or would only specific portions of the command document be generated from the JSON? This implies that there is some mechanism to merge the generated portions with the hand-written portions. How would this work? Exactly which portions would be generated?
* Generate the commands table which currently resides in server.c?
* I assume that we would remove this table from server.c and move it into a separate file which is completely auto-generated?
* The JSON would have to encode things like the various command flags.
* How would the generation of the new .c file occur? What are the implications on build?
Comment From: yoav-steinberg
All the commands above, excluding COMMAND, have no meaning with a subcommand. That means we will need to represent both COMMAND and COMMAND INFO, etc. in the command table.
Didn't you mean "have no meaning without a subcommand"?
Comment From: yoav-steinberg
Some keys cannot be found easily (KEYS in MIGRATE: Imagine the argument for AUTH is the string “KEYS” - we will start searching in the wrong index). The guarantee is that the specs may be incomplete (incomplete will be specified in the spec to denote that) but we never report false information.
I didn't quite understand the idea of "incomplete", how would we detect we can't figure out which are the valid keys here?
The only way to handle this is to know there's an arg called AUTH which expects a single argument and then to ignore that argument when searching for KEYS. This makes me wonder why not use the new commands.json format's arguments spec to "know" which arguments a command has and which of them are keys? Then we won't need the key_specs part?
Regarding the "tips" feature. My feeling is that the redis command developer can't really know what the client-lib/proxy/whoever wants to do with the command. In most cases the proxy can assume that if a command has keys it should send it to the relevant shard and if it doesn't have any keys it should distribute it to all shards. Regarding how to treat the response of a distributed command (sum, average, fail if single command fails or success if single command succeeds, etc.) this is entirely up to the implementer of the client/proxy and there might be different thing's you'd want to do in different cases. For example it makes sense to sum on DBSIZE, but there's also value for avg or even to fail the command. This depends on the proxy. I'd suggest one of the following
1) Don't include tips at all (my preferred solution).
2) Add a mechanism for configuring tips per command. We can even have default tips in the commands.json and generate a tips.conf (with a -include tips.conf in the default redis.conf). This way tips are a configuration which can be changed according to your needs.
Comment From: guybe7
@JimB123
Generate the documentation at https://redis.io/commands/ Would the entire command document be generated from the JSON? This implies that a large quantity of text documentation would be included in the JSON. Or would only specific portions of the command document be generated from the JSON? This implies that there is some mechanism to merge the generated portions with the hand-written portions. How would this work? Exactly which portions would be generated?
the latter. even now the information in redis.io comes from different sources (the page body, describing the command, comes from <command name>.md, and the rest comes from the command section in commands.json)
the idea is to use Redis to build the new commands.json for redis.io to use (the long command description, held in <command name>.md will remain as-is inside the redis-doc repo.
Generate the commands table which currently resides in server.c? I assume that we would remove this table from server.c and move it into a separate file which is completely auto-generated? The JSON would have to encode things like the various command flags. How would the generation of the new .c file occur? What are the implications on build?
yes, we will generate a new file, commands.c, which has the entire command table (will be an extern in server.c)
we will have a build target that depends on all the per-command json files. if any of the changes, it re-generates commands.c.
these pre-command json files will contain everything there's to know about a command (except for the long command description, which will remain in redis-doc)
Comment From: guybe7
@yoav-steinberg
Didn't you mean "have no meaning without a subcommand"?
yes, updated
I didn't quite understand the idea of "incomplete", how would we detect we can't figure out which are the valid keys here? The only way to handle this is to know there's an arg called AUTH which expects a single argument and then to ignore that argument when searching for KEYS. This makes me wonder why not use the new commands.json format's arguments spec to "know" which arguments a command has and which of them are keys? Then we won't need the key_specs part?
we already know the specs are limited and can't describe some schemes. the most complete solution is to describe each command's arg using some grammar and then the client can use this grammar to parse args and deduce key positions but we dropped it since we thought it's gonna be too complex for client writers and thus go unused.
the purpose if "incomplete" is to tell the client writer that if he wants the keys described in this specific spec he must use COMMAND GETKEYS(and then will receive all keys). it is possible for COMMAND GETKEYS to use the command arguments described in MIGRATE, SORT and STRALGO LCS)
Regarding the "tips" feature. My feeling is that the redis command developer can't really know what the client-lib/proxy/whoever wants to do with the command. In most cases the proxy can assume that if a command has keys it should send it to the relevant shard and if it doesn't have any keys it should distribute it to all shards. Regarding how to treat the response of a distributed command (sum, average, fail if single command fails or success if single command succeeds, etc.) this is entirely up to the implementer of the client/proxy and there might be different thing's you'd want to do in different cases. For example it makes sense to sum on DBSIZE, but there's also value for avg or even to fail the command. This depends on the proxy. I'd suggest one of the following
I feel like we would like to have this mechanism and I do like proposal number 2: having configurable tips. I'll update the description with that propsal
Comment From: zuiderkwast
the page body, describing the command, comes from
<command name>.json
@guybe7 I think you mean <command name>.md.
yes, we will generate a new file, commands.c, which has the entire command table (will be an extern in server.c) we will have a build target that depends on all the per-command json files.
This implies there will be a new build dependency which does the JSON processing. I suggest you pick a simple one which is already installed in most environments, such as a Python script using the built-in json package.
Comment From: guybe7
@zuiderkwast
thanks, updated
yes, probably python
Comment From: madolson
I think this is a great initiative!
Regarding clients, there is a pretty big emphasis on interpreted languages that don't rely on static typing. I think we should equally consider languages like Rust to be a first class support, so that they can directly consume this file or supporting files during compilation. In that vein, I wanted to put a greater emphasis on us getting "return_types" to be right.
Your example:
"return_types": { "2": ["+OK", "<bulk-string>", "<null-bulk-string>"], "3": ["+OK", "<bulk-string>", "<null>"], },
+OK isn't a type. So we should only be return the real types that are outlined in the RESP protocol.
Comment From: oranagra
I think we should equally consider languages like Rust to be a first class support, so that they can directly consume this file or supporting files during compilation. In that vein, I wanted to put a greater emphasis on us getting "return_types" to be right.
One of my objectives for pushing this project forward is that clients won't need to know the commands at compile time or coding time (similar to one of RESP3 goals).
This way a client that has an execute_command(str) concept will support new commands that are part of a new redis release implicitly, and also module commands will be supported the same way (will get closer to being first class citizens).
I know this docents come in place of real command wrappers in non-dynamic languages, but I Anna stress that my main aim is for clients to consume that Metadata from Redis's COMMAND command at runtime. (it'll generate a similar json format or RESP output), not take it from Redis source tree..
Regarding return_types it's right that +OK isn't a type. In redis.io it says the type is simple-string but it also says that the content of that string is +OK. In fact redis doesn't really have any other simple-string type returns (except for other special success statuses, like +CONTINUE).
So I'm looking for something that will both describe the type, but will also make it clear what it means (don't want just a plain simple-string and bulk-string one next to the other, it's misleading)
Comment From: madolson
I don't see the harm of having statically typed and compiled languages using the Redis source to build out the clients. I understand that it is less elegant, but I personally have a very strong preference very strong typing since I think it greatly helps developers write correct code. If I'm writing a rust library that discovers the commands at startup, I still have to do client.execute_command("Whatever"), and even though the client can route it, you still have to do ugly rust stuff to unwrap the type.
I also think it's naive for us to just assume we will have infinitely forward compatible clients, since we will probably introduce new functionality over time.
Related to string type. In other type systems you might introduce constraints. In redis for example, we could have something like:
"return_types": {
"2": [
{"type": "<simple-string>",
"constraint": {
"type": "equals",
"value": "+OK", }
},
"summary": "Command executed successfully"
{ type: "<null-bulk-string>",
"constraint": NULL,
"summary": "Command failed"
}
...
},
We can even go so far as to include what each constraint means, so we can drop the "return_summary" value and just generate it. The other approach I suppose as you mentioned would be to introduce a lot of psuedo types like OK/CONTINUE/positive-integer,etc.
Comment From: zuiderkwast
The simple strings are always fixed strings and they're used as tags, rather than as generic strings. I think we shouldn't give too much attention to the fact that they're called "strings". They're more like the LISP's symbols or Erlang's atoms than regular strings.
As tags, they are used to distinguish the return value from an error or another return value. In Rust's type system, they serve the same purpose as the variant name in an enum (called algebraic datatypes (ADTs) or tagged unions in other languages).
enum ResultOfSetCommand {
Ok,
OldValue(String),
NoOldValue,
}
(Related: In RESP errors, the first word of the error (ERR, WRONGTYPE, MOVED, etc.) also serves as a tag, but errors are not per command and this tag is an indication to the client rather than to the user.)
Comment From: itamarhaber
A suggestion by @aviavni: add a flag to indicate deprecation (e.g. HMSET -> "deprecated": "4.0.0")
Comment From: itamarhaber
Also, history should be something like:
{
"history": [
["since-version", "change-text"]
]
}
Comment From: itamarhaber
Yet another related suggestion: since is just the special case of the first, mandatory entry in the history of a command/modifier/argument. Put differently, every command/subcommand (and possibly argument/modifier) must have a history, the first entry of which is the version it was introduced. For args/mods, this can default to the command's history 1st entry unless manually specified otherwise
Comment From: itamarhaber
More half-thought-of suggestions:
- Arguments can have an optional
historyanddeprecated. - Arguments (arguably only oneof and block) should have a way to designate the default, if it exists (e.g. the
=in trimming policies). - Arguments, key-specs should allow an optional description
- Return_types should have a mandatory description of the value (theoretically a decomposition of the summary and its obsolescence).
- Return_types can be nested (i.e. an array of arrays)
- Key-specs can(must?) be data-structure-typed (i.e.
<string>,<list>, ...,<stream>,<any>, or a subset of these) - There needs to be an easy way to connect a key-spec and its respective argument (e.g. by
name?) - ...and maybe a way to tie key-specs, wait for it..., to complexity :)
Comment From: itamarhaber
I've started putting together a schema of the above. Ultimately it should belong to the redis org, but the WIP is at https://github.com/itamarhaber/redis-commands-schema
Comment From: zuiderkwast
I can't find this JSON format documented anywhere, apart from in this issue. Can we add it to the redis.io website?
The motivation is compile time support in compiled cluster clients. Similar JSON files are provided in modules like RediSearch and RedisJSON, with small differences such key specs are lacking, etc.
In a client lib (hiredis-cluster) we use the JSON files to build a command table prior to compile time, i.e. it's a header file checked in into the repo generated from the JSON files using a Python script, much like how we do it in Redis itself. If users need support for custom commands (modules) they can add additional JSON files and regenerate the header file.
Comment From: oranagra
i don't think we want to document this JSON format in redis.io (we can document it in a README inside the commands folder.
These JSON files should not be used anywhere outside redis itself (that can be mentioned in the README too).
when redis reads them (or actually processes the generated commands.def), there are some implicit fields that are added.
all external users should be relying on the output of COMMAND INFO and COMMAND DOCS, and if they want, they can convert that to JSON using redis-cli --json, same as utils/generate-commands-json.py does.
this format is arguably already documented in the docs of the COMMAND command.
is that sufficient? do you wanna draft the above mentioned README file?
Comment From: zuiderkwast
In compile time, it's convenient to use the JSON source files rather than fetching the information from a running Redis instance, not least when you want your compiled client to support not only that particular running instance. (I noticed when running in cluster mode, the sentinel commands don't exist, etc.)
all external users should be relying on the output of
COMMAND INFOandCOMMAND DOCS, and if they want, they can convert that to JSON usingredis-cli --json, same asutils/generate-commands-json.pydoes. this format is arguably already documented in the docs of the COMMAND command.
The generate-commands-json.py does considerable manipulation of the redis-cli --json output, so the output is actually more similar to the source files again, though not identical. Thus, we have at least three forms of JSON.
I'm now aiming to support both forms of JSON in our client's pre-compile script (the source files and the generate-commands-json.py output), since the two formats are similar enough. (We only need to find the first key for cluster routing, so we only use key specs with a best-effort fallback to arguments if key specs are not present.)
do you wanna draft the above mentioned README file?
That's not my no. 1 priority. :)
Comment From: oranagra
maybe you should be getting the json file from redis-doc then, instead of the redis code. one advantage is that like you said, they contain all the commands, including sentinel and cluster, etc. the second advantage is that it doesn't rely on the internal json files (which are missing some implicit attributes that are added by the redis code), you're actually using the output of COMMAND. and the last advantage is that you're logically using the official documentation to generate your code (it seems like the right thing)
@guybe7 maybe you can draft the README i mentioned in my previous comment?
Comment From: zuiderkwast
maybe you should be getting the json file from redis-doc then, instead of the redis code. one advantage is that like you said, they contain all the commands, including sentinel and cluster, etc.
@oranagra Good idea, but we noticed the SENTINEL commands are missing in redis-doc's JSON file. Actually they don't show up under https://redis.io/commands/, which may be a problem for the redis.io website...
Comment From: oranagra
:man_facepalming: i was writing this from memory, for some reason i remembered that this script runs redis in several ways and combines the commands into one json. well, at least this wrong statement didn't make it to #13066.
so now i wonder how did the CLUSTER commands make it to the redis.io json file. i realize that we're not filtering them from COMMAND, even though the instance doesn't really support them. well, we have room for improvement in both of these areas (COMMAND filtering, and redis.io json generation).
at least the new readme brings us to a slightly better place than before and doesn't contain wrong info :smile: