redis build on vultr's VPS os: ubuntu 18.04 LTS MemTotal: 492952 kB
Is memeory too small?
=== REDIS BUG REPORT START: Cut & paste starting from here ===
22831:M 09 Oct 01:36:42.315 # ------------------------------------------------
22831:M 09 Oct 01:36:42.315 # !!! Software Failure. Press left mouse button to continue
22831:M 09 Oct 01:36:42.315 # Guru Meditation: Redis aborting for OUT OF MEMORY #server.c:3594
22831:M 09 Oct 01:36:42.315 # (forcing SIGSEGV in order to print the stack trace)
22831:M 09 Oct 01:36:42.315 # ------------------------------------------------
22831:M 09 Oct 01:36:42.315 # Redis 4.0.11 crashed by signal: 11
22831:M 09 Oct 01:36:42.315 # Crashed running the instruction at: 0x558dbaac9365
22831:M 09 Oct 01:36:42.315 # Accessing address: 0xffffffffffffffff
22831:M 09 Oct 01:36:42.315 # Failed assertion:
------ STACK TRACE ------ EIP: src/redis-server 127.0.0.1:21379(_serverPanic+0x145)[0x558dbaac9365]
Backtrace: src/redis-server 127.0.0.1:21379(logStackTrace+0x5a)[0x558dbaacb03a] src/redis-server 127.0.0.1:21379(sigsegvHandler+0xb1)[0x558dbaacb7f1] /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7f7911a1a890] src/redis-server 127.0.0.1:21379(_serverPanic+0x145)[0x558dbaac9365] src/redis-server 127.0.0.1:21379(redisOutOfMemoryHandler+0x34)[0x558dbaa86f24] src/redis-server 127.0.0.1:21379(zmalloc+0x24)[0x558dbaa90684] src/redis-server 127.0.0.1:21379(createReplicationBacklog+0x1a)[0x558dbaaa190a] src/redis-server 127.0.0.1:21379(syncCommand+0x25c)[0x558dbaaa57cc] src/redis-server 127.0.0.1:21379(call+0xab)[0x558dbaa8939b] src/redis-server 127.0.0.1:21379(processCommand+0x48f)[0x558dbaa89bbf] src/redis-server 127.0.0.1:21379(processInputBuffer+0x115)[0x558dbaa99c65] src/redis-server 127.0.0.1:21379(aeProcessEvents+0x150)[0x558dbaa83090] src/redis-server 127.0.0.1:21379(aeMain+0x2b)[0x558dbaa8347b] src/redis-server 127.0.0.1:21379(main+0x4ec)[0x558dbaa8004c] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7f7911638b97] src/redis-server 127.0.0.1:21379(_start+0x2a)[0x558dbaa8027a]
------ INFO OUTPUT ------
Server
redis_version:4.0.11 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:1824731e954d92a4 redis_mode:standalone os:Linux 4.15.0-23-generic x86_64 arch_bits:64 multiplexing_api:epoll atomicvar_api:atomic-builtin gcc_version:7.3.0 process_id:22831 run_id:430fe7a734d5985f6cc7aaa363fe401584add0aa tcp_port:21379 uptime_in_seconds:1 uptime_in_days:0 hz:10 lru_clock:12322346 executable:/home/chenbin/download/redis-4.0.11/src/redis-server config_file:/home/chenbin/download/redis-4.0.11/./tests/tmp/redis.conf.20967.36
Clients
connected_clients:4 client_longest_output_list:0 client_biggest_input_buf:0 blocked_clients:0
Memory
used_memory:1104898 used_memory_human:1.05M used_memory_rss:4726784 used_memory_rss_human:4.51M used_memory_peak:1104898 used_memory_peak_human:1.05M used_memory_peak_perc:105.23% used_memory_overhead:1074821 used_memory_startup:852747 used_memory_dataset:30077 used_memory_dataset_perc:11.93% total_system_memory:504782848 total_system_memory_human:481.40M used_memory_lua:37888 used_memory_lua_human:37.00K maxmemory:0 maxmemory_human:0B maxmemory_policy:noeviction mem_fragmentation_ratio:4.28 mem_allocator:libc active_defrag_running:0 lazyfree_pending_objects:0
Persistence
loading:0 rdb_changes_since_last_save:283 rdb_bgsave_in_progress:0 rdb_last_save_time:1539049001 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:-1 rdb_current_bgsave_time_sec:-1 rdb_last_cow_size:0 aof_enabled:0 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_last_write_status:ok aof_last_cow_size:0
Stats
total_connections_received:6 total_commands_processed:946 instantaneous_ops_per_sec:517 total_net_input_bytes:72731 total_net_output_bytes:10098 instantaneous_input_kbps:39.66 instantaneous_output_kbps:5.07 rejected_connections:0 sync_full:1 sync_partial_ok:0 sync_partial_err:1 expired_keys:0 expired_stale_perc:0.00 expired_time_cap_reached_count:0 evicted_keys:0 keyspace_hits:257 keyspace_misses:242 pubsub_channels:0 pubsub_patterns:0 latest_fork_usec:0 migrate_cached_sockets:0 slave_expires_tracked_keys:0 active_defrag_hits:0 active_defrag_misses:0 active_defrag_key_hits:0 active_defrag_key_misses:0
Replication
role:master connected_slaves:1 slave0:ip=127.0.0.1,port=21380,state=wait_bgsave,offset=0,lag=0 master_replid:8deacb7e20ae6d69e174d4b373fc87daaff7d871 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:100000000 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0
CPU
used_cpu_sys:0.03 used_cpu_user:0.00 used_cpu_sys_children:0.00 used_cpu_user_children:0.00
Commandstats
cmdstat_lpush:calls=36,usec=228,usec_per_call=6.33 cmdstat_lpop:calls=6,usec=42,usec_per_call=7.00 cmdstat_config:calls=4,usec=15,usec_per_call=3.75 cmdstat_srem:calls=13,usec=51,usec_per_call=3.92 cmdstat_hdel:calls=23,usec=105,usec_per_call=4.57 cmdstat_replconf:calls=2,usec=3,usec_per_call=1.50 cmdstat_hset:calls=56,usec=348,usec_per_call=6.21 cmdstat_sunionstore:calls=7,usec=70,usec_per_call=10.00 cmdstat_zadd:calls=43,usec=630,usec_per_call=14.65 cmdstat_type:calls=499,usec=498,usec_per_call=1.00 cmdstat_sinterstore:calls=6,usec=36,usec_per_call=6.00 cmdstat_del:calls=34,usec=35,usec_per_call=1.03 cmdstat_zrem:calls=9,usec=42,usec_per_call=4.67 cmdstat_rpop:calls=6,usec=39,usec_per_call=6.50 cmdstat_zunionstore:calls=3,usec=72,usec_per_call=24.00 cmdstat_zinterstore:calls=7,usec=72,usec_per_call=10.29 cmdstat_sadd:calls=62,usec=250,usec_per_call=4.03 cmdstat_select:calls=4,usec=3,usec_per_call=0.75 cmdstat_sdiffstore:calls=4,usec=31,usec_per_call=7.75 cmdstat_set:calls=28,usec=128,usec_per_call=4.57 cmdstat_rpush:calls=6,usec=23,usec_per_call=3.83 cmdstat_ping:calls=2,usec=1,usec_per_call=0.50 cmdstat_randomkey:calls=84,usec=132,usec_per_call=1.57 cmdstat_lrem:calls=2,usec=14,usec_per_call=7.00
Cluster
cluster_enabled:0
Keyspace
db9:keys=38,expires=0,avg_ttl=0 db11:keys=48,expires=0,avg_ttl=0 db12:keys=43,expires=0,avg_ttl=0
------ CLIENT LIST OUTPUT ------ id=4 addr=127.0.0.1:44341 fd=7 name= age=1 idle=1 flags=N db=9 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=config id=5 addr=127.0.0.1:45253 fd=8 name= age=1 idle=0 flags=N db=9 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=type id=6 addr=127.0.0.1:46179 fd=9 name= age=1 idle=0 flags=N db=11 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=type id=7 addr=127.0.0.1:46021 fd=10 name= age=1 idle=0 flags=N db=12 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=type id=8 addr=127.0.0.1:39077 fd=11 name= age=0 idle=0 flags=S db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=psync
------ CURRENT CLIENT INFO ------ id=8 addr=127.0.0.1:39077 fd=11 name= age=0 idle=0 flags=S db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=psync argv[0]: 'PSYNC' argv[1]: '05a47094d35a5e06274474ed881b5da2737ae5a4' argv[2]: '1'
------ REGISTERS ------ 22831:M 09 Oct 01:36:42.320 # RAX:0000000000000000 RBX:00007ffdf5905c50 RCX:00007f7911727187 RDX:0000000000000000 RDI:0000000000000000 RSI:00007f7911a048c0 RBP:0000558dbab23e25 RSP:00007ffdf5905c30 R8 :0000000000000000 R9 :00007f79123e5b80 R10:00000000ffffffd0 R11:0000000000000000 R12:0000000000000e0a R13:000000000000000f R14:000577c1c5132d83 R15:00007ffdf5905ec0 RIP:0000558dbaac9365 EFL:0000000000010206 CSGSFS:002b000000000033 22831:M 09 Oct 01:36:42.320 # (00007ffdf5905c3f) -> 00007f7908000020 22831:M 09 Oct 01:36:42.320 # (00007ffdf5905c3e) -> 0000000005f5e110 22831:M 09 Oct 01:36:42.320 # (00007ffdf5905c3d) -> 00007f7908000020 22831:M 09 Oct 01:36:42.320 # (00007ffdf5905c3c) -> b9ed8b57bc10e5b1 22831:M 09 Oct 01:36:42.320 # (00007ffdf5905c3b) -> 7306bdb0e3448d81 22831:M 09 Oct 01:36:42.320 # (00007ffdf5905c3a) -> 698268b3bd55f307 22831:M 09 Oct 01:36:42.320 # (00007ffdf5905c39) -> b536caae85eb43f5 22831:M 09 Oct 01:36:42.320 # (00007ffdf5905c38) -> d7e7224500000000 22831:M 09 Oct 01:36:42.320 # (00007ffdf5905c37) -> 59524f4d454d2046 22831:M 09 Oct 01:36:42.320 # (00007ffdf5905c36) -> 4f2054554f20726f 22831:M 09 Oct 01:36:42.320 # (00007ffdf5905c35) -> 6620676e6974726f 22831:M 09 Oct 01:36:42.320 # (00007ffdf5905c34) -> 6261207369646552 22831:M 09 Oct 01:36:42.320 # (00007ffdf5905c33) -> 00007ffdf5905d60 22831:M 09 Oct 01:36:42.320 # (00007ffdf5905c32) -> 00007ffdf5905e30 22831:M 09 Oct 01:36:42.320 # (00007ffdf5905c31) -> 0000003000000018 22831:M 09 Oct 01:36:42.320 # (00007ffdf5905c30) -> ffffffffffffffff
------ FAST MEMORY TEST ------ 22831:M 09 Oct 01:36:42.320 # Bio thread for job type #0 terminated 22831:M 09 Oct 01:36:42.321 # Bio thread for job type #1 terminated 22831:M 09 Oct 01:36:42.329 # Bio thread for job type #2 terminated *** Preparing to test memory region 558dbad67000 (90112 bytes) *** Preparing to test memory region 558dbbc7c000 (1015808 bytes) *** Preparing to test memory region 7f7908000000 (135168 bytes) *** Preparing to test memory region 7f790faec000 (8388608 bytes) *** Preparing to test memory region 7f79102ed000 (8388608 bytes) *** Preparing to test memory region 7f7910aee000 (8388608 bytes) *** Preparing to test memory region 7f7911a04000 (16384 bytes) *** Preparing to test memory region 7f7911c23000 (16384 bytes) *** Preparing to test memory region 7f7912395000 (344064 bytes) *** Preparing to test memory region 7f79123f2000 (4096 bytes) .O.O.O.O.O.O.O.O.O.O Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.
------ DUMPING CODE AROUND EIP ------ Symbol: _serverPanic (base: 0x558dbaac9220) Module: src/redis-server 127.0.0.1:21379 (base 0x558dbaa5c000) $ xxd -r -p /tmp/dump.hex /tmp/dump.bin $ objdump --adjust-vma=0x558dbaac9220 -D -b binary -m i386:x86-64 /tmp/dump.bin
22831:M 09 Oct 01:36:45.184 # dump of function (hexdump of 453 bytes): 4154554189f4534889fd4881ece001000084c048898c24480100004c898424500100004c898c245801000074400f298424600100000f298c24700100000f299424800100000f299c24900100000f29a424a00100000f29ac24b00100000f29b424c00100000f29bc24d001000064488b042528000000488984242801000031c0488d842400020000488d5c24204c8d4c24084989d0b900010000ba010000004889442410488d842430010000be000100004889dfc744240818000000c744240c300000004889442418e8a25efbff8b05e02b2b0085c07505e8c3f7ffff488d35b46f060031c0bf03000000e8d0dafbff488d35d96f060031c0bf03000000e8bddafbff488d35908006004589e04889e94889da31c0bf03000000e8a1dafbff488d35ea6f060031c0bf03000000e88edafbff488d355f6f060031c0bf03000000e87bdafbffc60425ffffffff78488b842428010000644833042528000000750c4881c4e00100005b5d415cc3e85f62fbff0f1f440000662e0f1f8400000000004157660fefc041564155415455534881ecc8010000c747100000000064488b04252800000048898424b801000031c00f11078b0520262b0048897c241885c00f8e02090000 Function at 0x558dbaa86de0 is serverLog
=== REDIS BUG REPORT END. Make sure to include from START to END. ===
Comment From: yoav-steinberg
@robotstack Does this recreate? Can you pinpoint on what test it crashes? Since this is part of the CI and seems fine now I'm inclined to closing this. Please let me know if this is still relevant.