Describe the bug

Ghost nodes occur in the cluster and DNS reverse lookup is performed at the same time, causing performance problems

version 6.0.9

6b026fbd:0@0 fail? - 1626879202865 0 0 disconnected 7bbc2cd5:0@0 fail? - 1626879204870 0 0 disconnected

cluster forget 7bbc2cd5 (error) ERR Unknown node 7bbc2cd5

tcpdump [bad udp cksum 0x7ea6 -> 0x5ac4!] 23007+ AAAA? 6b026fbd. (26)

~]$ time /apps/svr/redis-6.0.14/bin/redis-cli -h x.x.x.x -p 8049 slowlog get 1 ^[[A1) 1) (integer) 16 2) (integer) 1626934246 3) (integer) 23548 4) 1) "cluster" 2) "nodes" 5) "x.x.x.x:49508" 6) ""

real 0m1.581s 《------- too slow user 0m0.000s sys 0m0.003s

How do I delete a ghost node？

Comment From: wonderful1984

Ipv6 is not enabled

Comment From: wonderful1984

redis-cli -p 8049 cluster nodes|grep -v fail|wc -l 512

128 source 384 replica

Server

redis_version:6.0.9 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:6616f483785125d redis_mode:cluster os:Linux 4.18.0-240.el8.x86_64 x86_64 arch_bits:64 multiplexing_api:epoll atomicvar_api:atomic-builtin gcc_version:8.4.0

Comment From: imzye

may I ask the nodeId 6b026fbd changed? try to forget it from all the instance simultaneously?

Comment From: wonderful1984

I can't forget these two nodes.

(error) ERR Unknown node

In the K8s environment, the container will be destroyed and the nodes.conf file will be lost. I will add Nodes and forget the fault container.

There's a chance that a ghost node will appear.

Comment From: 825644691

I also meet this problem, when I forget abnormal nodeid (Less than 40 ) ，the error 'Unknown node' occur

`/ CLUSTER FORGET / clusterNode *n = clusterLookupNode(c->argv[2]->ptr);

if (!n) { addReplyErrorFormat(c,"Unknown node %s", (char*)c->argv[2]->ptr); return; }`

typedef struct dictEntry { void *key; union { void *val; uint64_t u64; int64_t s64; double d; } v; struct dictEntry *next; } dictEntry;

`clusterNode clusterLookupNode(const char name) { sds s = sdsnewlen(name, CLUSTER_NAMELEN); dictEntry *de;

de = dictFind(server.cluster->nodes,s);
sdsfree(s);
if (de == NULL) return NULL;
return dictGetVal(de);

}`

`dictEntry dictFind(dict d, const void *key) { ...... while(he) {

        if (key==he->key || dictCompareKeys(d, key, he->key))
            return he;
        he = he->next;
    }
    if (!dictIsRehashing(d)) return NULL;
}
return NULL;

}`

it is possible "forget $node_id" $node_id and all the nodeid in redis fail to match or the dictGetVal() is ''?

but I use cluster nodes command,the abnormal nodes id display normally

cluster nodes command `sds clusterGenNodesDescription(int filter, int use_pport) { sds ci = sdsempty(), ni; dictIterator di; dictEntry de;

/* Generate all nodes slots info firstly. */
clusterGenNodesSlotsInfo(filter);

di = dictGetSafeIterator(server.cluster->nodes);
while((de = dictNext(di)) != NULL) {
    clusterNode *node = dictGetVal(de);

}` I discover redis loop by itself can get the abnormal nodes,

How did this mistake happen？