The problem/use-case that the feature addresses

Does Redis, inside a pipeline, allow a command to use values from a previous command within the same pipeline?

if not, to make a command depend on another command's output, redis needs to make multiple round trips to the database, or user must write lua.

network lag can paradoxically make a fast DB like Redis slower than a relational DB like PostgreSQL, because SQL can do in 1 round trip what requires many trips in Redis.

Description of the feature

I don't care about semantics, but it would be powerful if we could tell Redis to pass a command's result as input to another without network calls within one PIPE / EXECUTE transaction. This would enable a wide variety of uses, such as GraphQL or Relational / Set algebra operations, to happen in one round trip.

For example, say you wanted to get all the members of all the organizations of which one user is a member. Textbook case where SQL join or graph traversal is handy

with r.Pipe() as pipe:
    organization_ids = r.smembers(user_id + ":organizations")
    neighbor_ids = [pipe.smembers(organization_id + ":users") for organization_id in organization_ids]
    result = pipe.execute()

ideally, organization_ids shouldn't need to come back to python client, in order to run the neighbor_ids query

if the organization_ids thing were a placeholder (future?), or a node in a DAG, redis could immediately pass the value into the next command without touching the network

Alternatives you've considered

Lua can do this but requires context switch from client-side code into Lua. It would be cool if clients could tell redis to automatically pass the variables around.

8693 mentions functions, which could do this, but i wanted to suggest another alternative

we could use Postgres. But Redis data structures are awesome and deserve to be linked together!

Comment From: zuiderkwast

SQL is a powerful language, like Lua. SQL also requires a mental context switch from a client script language to SQL, so what's the difference?

Another way to do server-side logic is modules.

Does Redis, inside a pipeline, allow a command to use values from a previous command within the same pipeline?

Occasionally, a pipeline of commands like LMOVE, INCRBY and COPY combined with temporary keys can get the job done. For example, the optimistic locking transaction example from https://redis.io/topics/transactions can be done in a pipeline without extra round trips like this:

WATCH mykey
COPY mykey tmp
INCR tmp
MULTI
COPY tmp mykey REPLACE
DEL tmp
EXEC

(... where INCR tmp can be replaced with any other command operating on tmp ...)

The pipe code from your example could, in a language with a powerful macro system (like LISP), possibly, be rewritten to Lua code (by some domain specific language library) and sent to Redis behind the scenes, so it would fit better in to the client program. (I don't know of any such library, but I think it's possible.)

8693 suggests extensible support for more languages as you noted, but inventing our own language is an endless task. If we add pipes or variables, next you realize you can't do much without adding plus, minus, concatenation, comparisons, conditions, loops... until you end up with a full-fledged language, so why re-invent the wheel?

Comment From: madolson

Agree with @zuiderkwast, I think we should think deeply if we want to go down this route. We could consider optimizing for a couple of use cases, but I think LUA should be left as the way to do these types of operations. (Or modules)

Comment From: bionicles

ok, no problemo, thanks for the cool ideas re: lua codegen macros and tmp variables