Describe the bug
EVAL and EVALSHA do not support running lua byte code (.ljbc) and will result in error like
(error) ERR Error compiling script (new function): user_script:1: unexpected symbol near 'char(27)'
To reproduce
create byte code:
echo 'return redis.call("GET", KEYS[1])' > script.lua
luajit -b script.lua script.ljbc
redis-cli SET mykey "Hello"
redis-cli EVAL "$(<script.lua)" 1 mykey
redis-cli EVAL "$(<script.ljbc)" 1 mykey
Expected behavior
output "Hello" for both .lua and .ljbc
Additional information
This issue most likely also applies to the FUNCTION commands (as well as to EVAL_RO,... of course)
Comment From: sundb
@kkmuffme sorry, we don't support luajit yet, maybe you can try third party luajit for Redis.
Comment From: kkmuffme
It doesn't work with bytecode compiled with luac created bytecode either https://linux.die.net/man/1/luac which has been part of lua since ages.
echo 'return redis.call("GET", KEYS[1])' > script.lua
luac -o script.out script.lua
redis-cli SET mykey "Hello"
redis-cli EVAL "$(<script.lua)" 1 mykey
redis-cli --eval script.lua mykey
redis-cli --eval script.out mykey
Results in:
(error) ERR Error compiling script (new function): user_script:1: unexpected symbol near 'char(27)'
Comment From: sundb
@kkmuffme it's dangerous for Redis to load bytecode directly, which means we will lose control of all permissions.
Comment From: kkmuffme
Are you saying Redis parses the .lua to check for permissions before actually passing it to the lua interpreter/executing it?
Comment From: sundb
Are you saying Redis parses the .lua to check for permissions before actually passing it to the lua interpreter/executing it?
yes, I'm not sure if bytecode can bypass the prosecution to do something we're not allowed to do.
Comment From: kkmuffme
Are you talking about ACL related permission checks? (or validation the lua to ensure it's not breaking out of the sandbox?)
Once a script is loaded (and Redis has the sha1), is the lua stored as is or as bytecode? (= to avoid recompiling everytime)
If it's about ensuring it's not breaking out of the sandbox, I guess there should be an option to disable these checks to allow for additional performance when only handling static/trusted input.
Comment From: sundb
Are you talking about ACL related permission checks? (or validation the lua to ensure it's not breaking out of the sandbox?)
no, i'm talking about the lua sanbox.
Once a script is loaded (and Redis has the sha1), is the lua stored as is or as bytecode? (= to avoid recompiling everytime)
once the script is loaded, it will be stored as bytecode. but Function is different, now it's stored as bytes and a copy of the script is stored because it's datatype.
If it's about ensuring it's not breaking out of the sandbox, I guess there should be an option to disable these checks to allow for additional performance when only handling static/trusted input.
not sure if it's worth it, isn't it enough to just use load + evalsha?
Comment From: kkmuffme
not sure if it's worth it, isn't it enough to just use load + evalsha?
In most cases it is I guess. Since other parts of the environment we use Redis in uses lua bytecode (openresty/luajit), it's more about getting some minimal performance gains where they're easily achievable. Since Redis doesn't support that and it's a big effort to change that, I think it's not worth the time.
Comment From: sundb
not sure if it's worth it, isn't it enough to just use load + evalsha?
In most cases it is I guess. Since other parts of the environment we use Redis in uses lua bytecode (openresty/luajit), it's more about getting some minimal performance gains where they're easily achievable. Since Redis doesn't support that and it's a big effort to change that, I think it's not worth the time.
my main concern is security, openresty's lua script is provided by the developer, but redis's script is from users, and the lack of sandbox restrictions is dangerous.
Comment From: kkmuffme
openresty's lua script is provided by the developer redis's script is from users
Who allows/uses end-user input in redis scripts? (except the Redis demo page for the commands).
Usually the same developer who writes openresty lua code also writes the redis script lua.