The problem/use-case that the feature addresses

The most frequently used/demanded pattern about transactions is:

  • Check conditions
  • Execute transaction only when conditions satisfied
  • Do it in atomic way

Standard answer is:

  • Use lua scripting
  • Use WATCH

In other words, there are no ready human-friendly solution to use immediately :)

Description of the feature

Universal script, accepting conditions & transaction info as json object. See https://github.com/nodeca/redis-if for docs & samples.

That's battle-tested approach. Other custom scripts become not needed. Not tested with cluster, probably can be pinned to proper node by passing tag as key.

IMO this approach simplifies significantly most of edge cases, without need to write custom lua scripts. Including this into docs (transactions chapter), will help to many people.

Alternatives you've considered

WATCH - non-trivial & non-flexible

Additional information

If you find this useful, you can use materials of https://github.com/nodeca/redis-if for redis project needs as you wish, without referring to original.

Comment From: yossigo

@puzrin Thanks for raising this. It's not the first time I encounter a similar request, but I'm not sure I fully understand why this makes things easier. For example, why would this:

let success = await redis.transaction(JSON.stringify({
  if: [
    // apply changes only if this process has acquired a lock
    [ 'initialized', '==', [ 'sget', 'custom-state' ] ]
  ],
  exec: [
    [ 'set', 'custom-state', 'finished' ],
    [ 'incr', 'custom-counter' ]
  ]
}))

be simpler or easier to code than this:

let success = await redis.eval("\
    if redis.call('sget', 'custom-state') == 'initialized' then \
        redis.call('set', 'custom-state', 'finished') \
    else \
        redis.call('incr', 'custom-counter') \
    end", 0)

If it's a matter of handling the formatting, I think this task is really best left to client libraries to do in the most dogmatic form applicable to the language used. Otherwise I don't see a benefit here, but do see less expressive power. Am I too biased and locked in a paradigm here?

Comment From: puzrin

For example, why would this: be simpler or easier to code than this:

Answer is simple. Most of redis users do not know lua. They need to quickly express business logic and move forward.

It's not a big deal to learn lua, but without repeating practice you will forget everything after month. So, when you come from java/js/php/python/ruby, you have 2 problems:

  • more slow development (need to learn lua and switch "context of brain" between languages)
  • more difficult maintenance (scripts are readable, but need re-learn lua to modify).

Am I too biased and locked in a paradigm here?

I guess, you think like person who knows lua well :). And for your case writing custom scripts may be more optimal. But as i said, knowledge profiles of "redis developper" and "redis user" are different.

From my experience, concept of "conditional transaction" covers all or almost all cases when people are "forced" to use lua. I'd prefer have this feature out of box, like MULTI, but started proposal with more realistic approach - "one custom script for all edge cases".

Comment From: yossigo

@puzrin To be honest I do tend to forget Lua syntax and need to refresh once in a while, but luckily it's simple :) Your argument makes a lot of sense, but can it really be solved at the server side? This will still require users to express their conditionals in some Redis expression manner vs. expressing it locally to the client library which will provide the translation.

I'm not totally against this idea and as I mentioned it did come up in the past, but trying to very clearly define the value and understand if it's anything more than syntactic sugar.

Comment From: puzrin

I'm not sure i understand you right. Core problem is lack of feature "atomic check condition before transaction run". The only way to implement this right now - lua script. Lua script can be only server side. I stay aside area with locks requests/releases - that's much more heavy.

  • If your question is "Can ORM generate such script automatically" - may be. But ORM is very heavy layer, not for each case, language specific.
  • If you ask "can conditional transaction replace all lua script cases" - from my experience, this seems to replace need of lua script in 99% of cases. Using this approach is "natural" (does not break brain)

I'm not totally against this idea and as I mentioned it did come up in the past, but trying to very clearly define the value and understand if it's anything more than syntactic sugar.

Added value - remove ass pain for complex data flows design :). Need of postpone everything and interrupt for remember lua is very annoying (for programmer, for code reviewer, for maintainer)

Example of "complex data flow" - distributed task queue https://github.com/nodeca/idoit. With lot of lua script that would be nightmare.

This will still require users to express their conditionals in some Redis expression manner vs. expressing it locally to the client library which will provide the translation.

  1. Proposed format is carefully designed. It's much faster to write & to understand, than LUA.
  2. This is the first step. You can leave proposal as is (provided script), or can do it better (embed into redice with native syntax). I started in scope of existing possibilities - got great result with minimal efforts. That's already useable, but can be improved.

Comment From: yossigo

@puzrin What I meant was that if our goal is to let the transaction be expressed in a way that is as native as possible, then coming up with a native-client-specific (like you did for Nodejs) might end up better than designing a single core Redis command that will have its own expression representation.

@itamarhaber FYI, I think we discussed an IF/ELSE conditional commands recently.

Comment From: puzrin

What I meant was that if our goal is to let the transaction be expressed in a way that is as native as possible, then

I think, primary goal is to figure out, redis has not enougth [convenient] coverage for popular "conditional transaction" case. This should be solved SOMEHOW (approaches can be different).

  1. For constructive discussion, i provided script-based solution, useable immediately. Instead of abusing devs with asking for new feature :)
  2. While suggested solution is battle-tested, i think approach can be improved. Redis team has more experience to deside if it worth improve or not.
  • In minimal case - just add proposed approach into transaction docs. That will help a lot to many people.
  • In maximal case - i would trust to your opinion :). I don't know cost of native (alternate) implementation.