Crash report redis6_crash_report_May24.txt
May 23 21:22:37.231768+00:00 2024 leaf5 INFO database#supervisord: redis6 === REDIS BUG REPORT START: Cut & paste starting from here ===
May 23 21:22:37.232176+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.231 # Redis 6.2.6 crashed by signal: 11, si_code: 1
May 23 21:22:37.232600+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.232 # Accessing address: 0xffffffff830ff539
May 23 21:22:37.232749+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.232 # Crashed running the instruction at: 0x55e587d110f9
May 23 21:22:37.234150+00:00 2024 leaf5 INFO database#supervisord: redis6
May 23 21:22:37.234150+00:00 2024 leaf5 INFO database#supervisord: redis6 ------ STACK TRACE ------
May 23 21:22:37.234545+00:00 2024 leaf5 INFO database#supervisord: redis6 EIP:
May 23 21:22:37.234895+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(sdsMakeRoomFor+0x429)[0x55e587d110f9]
May 23 21:22:37.235203+00:00 2024 leaf5 INFO database#supervisord: redis6
May 23 21:22:37.235203+00:00 2024 leaf5 INFO database#supervisord: redis6 Backtrace:
May 23 21:22:37.235482+00:00 2024 leaf5 INFO database#supervisord: redis6 /lib/x86_64-linux-gnu/libpthread.so.0(+0x13140)[0x7f5a6dee7140]
May 23 21:22:37.235792+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(sdsMakeRoomFor+0x429)[0x55e587d110f9]
May 23 21:22:37.236126+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(sdscatlen+0x44)[0x55e587d11794]
May 23 21:22:37.236469+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(notifyKeyspaceEvent+0xd6)[0x55e587d89fe6]
May 23 21:22:37.236787+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(hsetCommand+0xf9)[0x55e587d51819]
May 23 21:22:37.237001+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(call+0xb1)[0x55e587d09bb1]
May 23 21:22:37.237323+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(luaRedisGenericCommand+0x3b9)[0x55e587d76df9]
May 23 21:22:37.237749+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(+0x117858)[0x55e587dd1858]
May 23 21:22:37.237993+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(+0x122020)[0x55e587ddc020]
May 23 21:22:37.238379+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(+0x117ef5)[0x55e587dd1ef5]
May 23 21:22:37.238674+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(+0x1171eb)[0x55e587dd11eb]
May 23 21:22:37.238994+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(+0x1180a0)[0x55e587dd20a0]
May 23 21:22:37.239335+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(lua_pcall+0x58)[0x55e587dcf468]
May 23 21:22:37.239686+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(evalGenericCommand+0x1ff)[0x55e587d743ff]
May 23 21:22:37.240007+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(call+0xb1)[0x55e587d09bb1]
May 23 21:22:37.240352+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(processCommand+0x5b3)[0x55e587d0aab3]
May 23 21:22:37.240687+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(processInputBuffer+0xf8)[0x55e587d1f738]
May 23 21:22:37.241010+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(+0x10513c)[0x55e587dbf13c]
May 23 21:22:37.241348+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(aeProcessEvents+0x2c2)[0x55e587d02032]
May 23 21:22:37.241695+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(aeMain+0x1d)[0x55e587d022bd]
May 23 21:22:37.242002+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(main+0x337)[0x55e587cfe307]
May 23 21:22:37.242311+00:00 2024 leaf5 INFO database#supervisord: redis6 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea)[0x7f5a6dd21d0a]
May 23 21:22:37.242637+00:00 2024 leaf5 INFO database#supervisord: redis6 /usr/bin/redis-server 127.0.0.1:63796(_start+0x2a)[0x55e587cfe83a]
May 23 21:22:37.242906+00:00 2024 leaf5 INFO database#supervisord: redis6
May 23 21:22:37.243008+00:00 2024 leaf5 INFO database#supervisord: redis6 ------ REGISTERS ------
May 23 21:22:37.243229+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.243 #
May 23 21:22:37.243383+00:00 2024 leaf5 INFO database#supervisord: redis6 RAX:ffffffff830ff539 RBX:0000000000000001
May 23 21:22:37.243383+00:00 2024 leaf5 INFO database#supervisord: redis6 RCX:000055e587e5d7e0 RDX:000055e587d10df1
May 23 21:22:37.243383+00:00 2024 leaf5 INFO database#supervisord: redis6 RDI:00007f5a6abb0001 RSI:0000000000000001
May 23 21:22:37.243558+00:00 2024 leaf5 INFO database#supervisord: redis6 RBP:00007f5a6abb0001 RSP:00007ffd00e4eda0
May 23 21:22:37.243558+00:00 2024 leaf5 INFO database#supervisord: redis6 R8 :000000007cf00ac8 R9 :00007ffd00e4ee40
May 23 21:22:37.243558+00:00 2024 leaf5 INFO database#supervisord: redis6 R10:0000000000000000 R11:0000000000000032
May 23 21:22:37.243766+00:00 2024 leaf5 INFO database#supervisord: redis6 R12:0000000000000002 R13:000000000000000b
May 23 21:22:37.243766+00:00 2024 leaf5 INFO database#supervisord: redis6 R14:00007f5a6abb0001 R15:00007ffd00e4ee40
May 23 21:22:37.243766+00:00 2024 leaf5 INFO database#supervisord: redis6 RIP:000055e587d110f9 EFL:0000000000010206
May 23 21:22:37.243766+00:00 2024 leaf5 INFO database#supervisord: redis6 CSGSFS:002b000000000033
May 23 21:22:37.244601+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.243 # (00007ffd00e4edaf) -> 0000000000000002
May 23 21:22:37.244776+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.243 # (00007ffd00e4edae) -> 00007f5a6ad260a8
May 23 21:22:37.244776+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.243 # (00007ffd00e4edad) -> 0000000000000001
May 23 21:22:37.244776+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.243 # (00007ffd00e4edac) -> 00007f5a6ad260a8
May 23 21:22:37.244776+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.243 # (00007ffd00e4edab) -> 000055e587d11794
May 23 21:22:37.245069+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.243 # (00007ffd00e4edaa) -> 00007ffd00e4ee40
May 23 21:22:37.245069+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.243 # (00007ffd00e4eda9) -> 00007f5a6abb0001
May 23 21:22:37.245069+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.243 # (00007ffd00e4eda8) -> 000000000000000b
May 23 21:22:37.245069+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.243 # (00007ffd00e4eda7) -> 0000000000000002
May 23 21:22:37.245069+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.243 # (00007ffd00e4eda6) -> 00007ffd00e4ee40
May 23 21:22:37.245069+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.243 # (00007ffd00e4eda5) -> 0000000000000001
May 23 21:22:37.245069+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.243 # (00007ffd00e4eda4) -> 000000000000000b
May 23 21:22:37.245474+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.243 # (00007ffd00e4eda3) -> 23cd2b772c1c5d00
May 23 21:22:37.245474+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.243 # (00007ffd00e4eda2) -> 00007f5a6abb0000
May 23 21:22:37.245474+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.243 # (00007ffd00e4eda1) -> 00007ffd00e4ede0
May 23 21:22:37.245474+00:00 2024 leaf5 INFO database#supervisord: redis6 75:M 23 May 2024 21:22:37.243 # (00007ffd00e4eda0) -> 00007f5a6a6b0000
May 23 21:22:37.245474+00:00 2024 leaf5 INFO database#supervisord: redis6
Full crash report attached.
Comment From: sg893052
@oranagra @sundb Could you please share your valuable inputs on this? Thanks!
Comment From: sundb
@sg893052 can you give the notification config? or other special configs?
Comment From: sg893052
@sg893052 can you give the notification config? or other special configs? @sundb I have attached the redis.conf file docker-database-redis.conf.txt
Also, attaching the supervisord.conf file for how the different redis server instances are spawned inside docker-database container. The problem occurred at redis6 instance. docker-database-supervisord.conf.txt
Inside docker-database container:
root@sonic:/# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.1 33624 25988 pts/0 Ss+ Jun05 0:27 /usr/bin/python3 /usr/local/bin/supervisord
root 68 0.0 0.0 2480 1568 pts/0 S Jun05 0:00 sh /usr/bin/supervisor-proc-exit-listener --container-name database
root 70 0.5 0.1 122512 26116 pts/0 Sl Jun05 7:38 /usr/bin/redis-server 127.0.0.1:6379
root 71 0.3 0.1 78344 16620 pts/0 Sl Jun05 4:37 /usr/bin/redis-server 127.0.0.1:63792
root 75 0.1 0.1 65660 15768 pts/0 Sl Jun05 1:50 /usr/bin/redis-server 127.0.0.1:63793
root 83 0.1 0.0 62340 13544 pts/0 Sl Jun05 1:28 /usr/bin/redis-server 127.0.0.1:63794
root 90 2.5 0.1 69548 18556 pts/0 Sl Jun05 33:21 /usr/bin/redis-server 127.0.0.1:63796
root 97 0.1 0.0 59780 13652 pts/0 Sl Jun05 1:25 /usr/bin/redis-server 127.0.0.1:63970
root 239 0.0 0.0 220904 3804 pts/0 Sl Jun05 0:01 /usr/sbin/rsyslogd -n -iNONE
root 408 0.0 0.0 7788 3956 pts/1 Ss Jun05 0:00 bash
root 3745 0.0 0.2 683676 45832 pts/1 Sl+ Jun05 0:00 gdb /usr/bin/redis-server /core.redis-server.0.ba1a0b8780ea4774b75a47d5ce70532c.1253.1716499357000000
root 9524 1.0 0.0 7788 3768 pts/2 Ss 08:53 0:00 bash
root 9530 0.0 0.0 10384 3340 pts/2 R+ 08:53 0:00 ps aux
Comment From: oranagra
@sg893052 did it happen once? or does it reproduce. i see this is a rather old version (even for 6.2.x), skimming over the release notes i don't see anything obvious, but still maybe you should try to upgrade.