Hello, I was looking for a COPY command, which would essentially does the same thing MOVE command already does, but without deleting source key. I didn't found any docs about this. Does a command like this exists? If not, I would like to submit a pull request implementing this feature.

Comment From: itamarhaber

Hello @AlessandroSechi

There is indeed no such command. However. Before you implement and submit a PR for it, it would be nice to understand the use case for it. Personally, this is the first time I've heard of someone needing it.

Also, note that COPY-like functionality can be easily composed with a server-side Lua script such as this one:

redis.call('SELECT', ARGV[1])
local ser = redis.call('DUMP', KEYS[1])
redis.call('SELECT', ARGV[2])
redis.call('RESTORE', KEYS[2], 0, ser)

```

Comment From: AlessandroSechi

Hello @itamarhaber , thank you for your fast reply. I needed this command while I was doing some tests in my Redis setup. I had to duplicate a key for testing purposes, but I still had to keep its name and original content. So, the only option seemed to copy the key in another database, but using the MOVE command the original key would be lost.

Comment From: itamarhaber

Thanks for the background - it makes sense to need this in the context of testing. That said, in my opinion, DUMP and RESTORE are your friends here.

Comment From: AlessandroSechi

I just saw the documentation and I agree with you in my use case. But I also spotted in the DUMP documentation that the expire value is not “dumped”, so it will need another command to get the TTL, while the COPY command would be “one-line”, like the MOVE one . My purpose is merely intended as a “simplification” of the entire process

Comment From: madolson

I would like to revive this because i've also seen use cases for this, and it relates to other discussions about the restore command. The use case I heard about is that they were using it for checkpointing in a complex transaction system. They were cloning items and then making changes to it. These changes weren't immediate, but built up over time, and once all the checks passed they would replace the old item. They were doing dump/restore (I think it was hashes and sorted sets, but I didn't take note of that) in a lua command to achieve this.

My thoughts: 1. It's a great use case for the background threads since copying can be expensive for large values. 2. Dump/restore isn't atomic unless you do it in a lua script, which will lock everything. 3. It's inline with some of our other pushes for straight forward commands.

@redis/core-team Thoughts?

Comment From: yossigo

@madolson These are all valid and sensible arguments for it. The only argument against it is that multi-DB has its own share of issues, and as it's already omitted from Redis Cluster it should generally not be promoted.

Comment From: oranagra

maybe by default COPY or CLONE would create an additional copy of the key in the same database, and the multi-DB feature would be achieved with an optional additional argument? some random set of options:

  1. COPY <key> <new-key> <dest-db> [REPLACE]
  2. COPY <key> <new-key> [DB <dest-db>] [REPLACE]

or maybe by utilizing an existing command (ugly IMHO):

  1. MOVE <key> <db> [KEEP_EXISTING] - note this doesn't allow cloning in the same db to a new name
  2. MOVE <key> <db> [KEEP_EXISTING] [NEW_NAME <new-name>] - this is not a good option since it's hard to extract key names from the command.
  3. MIGRATE localhost localport <key> [COPY] [REPLACE] - i.e. we detect a migrate attempt to self, and instead of dump+restore do deep copy.

option 2 seems nice to me, but i'm not sure what are the implications of having a command in which a db index is an optional argument. i guess that in an instance were there's only one database, it doesn't really matters, so maybe that's ok.

Comment From: madolson

My initial thought was the same as @oranagra. Since you can still get the same guarantees with a COPY + MOVE and we could incrementally build the DB selection as well. Option 2 would also be my vote.

I also still see a lot of usage of DBs, I think just saying it's discouraged is sub-optimal since we don't really have a replacement. The lack of "FLUSHDB" in cluster mode is a noticeable gap I've seen come up time and again.

@AlessandroSechi Do you still have interest in implementing it? I don't think it will be trivial.

Comment From: madolson

Status: It looks like we have enough of a design to get started on the implementation. We can start with just the COPY to the same DB, and then evaluate and extend it from there.

The implementation here is non-trivial, as it requires concrete implementations for every data structure.

Comment From: tmgauss

Hello, @madolson

Actually, I am tackling this issue with my friends at university. We almost finished coding as follows

  • Make copyCommand func to db.c
  • Add copyCommand func to server.c and server.h

Then, I have two questions:

  1. Is it enough to copy data and notify other clients by signalModifiedKey func & notifyKeyspaceEvent func?
  2. Is it enought to add test units to tests/unit/keyspace.tcl?

Comment From: oranagra

@tmgauss the answer to your questions is "yes", but note that both the implementation and tests need to cover all data types (i.e. do a deep copy of everything).

Comment From: tmgauss

Thank you for your swift reply.

Okay, then I'll discuss more about how to implement the command with my teammates.

Comment From: madolson

Oran is always watching :D

A reminder that there is a DEBUG digest you can use to verify the keys: https://github.com/redis/redis/blob/unstable/src/debug.c#L388

@tmgauss How did you find and decide to work on the issue, if you don't mind me asking?

Comment From: tmgauss

@madolson Actually because we're taking the class where we contribute to OSS!!

Comment From: tmgauss

I'm glad to say that our team finished coding

  • Can copy strings
  • Can copy lists
  • Can copy sets
  • Can copy sorted sets
  • Can copy hash
  • Can copy stream
  • Test whether COPY command can copy data properly

Then what should we do next? Should we send PR or email to the management team?

Comment From: oranagra

@tmgauss thank you. Please submit a PR and we can continue the discussion and review there. P.s. I suppose module keys cannot work with this command unless we add specific module API for that. Also, I suppose it would be a good idea to try to run the tests with Valgrind.