We have an application that is under heavy load. ActivMQ and Redis are used. We use the standard RoR library "activejob-6.0.3.4" to write to Redis. The problem is that when the “workers” and “consumers” are running at the same time, a slow transfer to Redis begins. Delay time up to 3 minutes, average 4 seconds. When the “workers” are disabled, the ActivMQ queue quickly leaves. After turning on the workers, the queue accumulates again and Redis starts to work more slowly.

REDIS_1

Processing on ActivMQ. The red line is the normal for the processing to work.

Redis configs and requests between ActivMQ - Consumer - Redis are attached.

Comment From: zuiderkwast

Hi!

Redis configs and requests between ActivMQ - Consumer - Redis are attached.

I can't see any attached files.

Redis requests and config would be useful to see. Also some info such as Redis memory usage would be interesting to see. Which type of keys are used in Redis? Is it streams or pubsub or something else?

Comment From: olegredis

Hi Please see the attached files

From: Viktor Söderqvist @.> Sent: Friday, 11 November 2022 12:46 To: redis/redis @.> Cc: Oleg Kovalev @.>; Author @.> Subject: Re: [redis/redis] Slow Redis Active MQ (Issue #11495)

Hi!

Redis configs and requests between ActivMQ - Consumer - Redis are attached.

I can't see any attached files.

Redis requests and config would be useful to see. Also some info such as Redis memory usage would be interesting to see. Which type of keys are used in Redis? Is it streams or pubsub or something else?

— Reply to this email directly, view it on GitHubhttps://github.com/redis/redis/issues/11495#issuecomment-1311534361, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A4DW5UTONJJAJ3H63VJH5TDWHYPWRANCNFSM6AAAAAAR4VYSQQ. You are receiving this because you authored the thread.Message ID: @.**@.**>>

Server

redis_version:6.2.1 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:caae132ecaa7eab0 redis_mode:standalone os:Linux 3.10.0-1062.el7.x86_64 x86_64 arch_bits:64 multiplexing_api:epoll atomicvar_api:c11-builtin gcc_version:8.3.1 process_id:23736 process_supervised:systemd run_id:c0c2dec02e1cfd3c675a97f9cb64f42b530c96b5 tcp_port:6379 server_time_usec:1668084540144704 uptime_in_seconds:20049 uptime_in_days:0 hz:30 configured_hz:30 lru_clock:7140156 executable:/usr/bin/redis-server config_file:/etc/redis/redis.conf io_threads_active:0

Clients

connected_clients:386 cluster_connections:0 maxclients:35000 client_recent_max_input_buffer:1578 client_recent_max_output_buffer:1170351912 blocked_clients:0 tracking_clients:0 clients_in_timeout_table:0

Memory

used_memory:5011323280 used_memory_human:4.67G used_memory_rss:5872250880 used_memory_rss_human:5.47G used_memory_peak:21766933408 used_memory_peak_human:20.27G used_memory_peak_perc:23.02% used_memory_overhead:2625944808 used_memory_startup:2368424 used_memory_dataset:2385378472 used_memory_dataset_perc:47.62% allocator_allocated:5012481528 allocator_active:5620498432 allocator_resident:5982396416 total_system_memory:33567358976 total_system_memory_human:31.26G used_memory_lua:37888 used_memory_lua_human:37.00K used_memory_scripts:0 used_memory_scripts_human:0B number_of_cached_scripts:0 maxmemory:0 maxmemory_human:0B maxmemory_policy:noeviction allocator_frag_ratio:1.12 allocator_frag_bytes:608016904 allocator_rss_ratio:1.06 allocator_rss_bytes:361897984 rss_overhead_ratio:0.98 rss_overhead_bytes:-110145536 mem_fragmentation_ratio:1.17 mem_fragmentation_bytes:860975960 mem_not_counted_for_evict:0 mem_replication_backlog:1048576 mem_clients_slaves:61536 mem_clients_normal:2622212944 mem_aof_buffer:0 mem_allocator:jemalloc-5.1.0 active_defrag_running:0 lazyfree_pending_objects:0 lazyfreed_objects:0

Persistence

loading:0 current_cow_size:0 current_fork_perc:0.00% current_save_keys_processed:0 current_save_keys_total:0 rdb_changes_since_last_save:51822 rdb_bgsave_in_progress:0 rdb_last_save_time:1668084525 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:8 rdb_current_bgsave_time_sec:-1 rdb_last_cow_size:99733504 aof_enabled:0 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_last_write_status:ok aof_last_cow_size:0 module_fork_in_progress:0 module_fork_last_cow_size:0

Stats

total_connections_received:357758 total_commands_processed:311529878 instantaneous_ops_per_sec:5170 total_net_input_bytes:117857206564 total_net_output_bytes:533933684979 instantaneous_input_kbps:1789.32 instantaneous_output_kbps:26013.10 rejected_connections:0 sync_full:9 sync_partial_ok:0 sync_partial_err:7 expired_keys:8258 expired_stale_perc:0.00 expired_time_cap_reached_count:0 expire_cycle_cpu_milliseconds:14317 evicted_keys:0 keyspace_hits:37887103 keyspace_misses:303887 pubsub_channels:0 pubsub_patterns:0 latest_fork_usec:119321 total_forks:308 migrate_cached_sockets:0 slave_expires_tracked_keys:0 active_defrag_hits:0 active_defrag_misses:0 active_defrag_key_hits:0 active_defrag_key_misses:0 tracking_total_keys:0 tracking_total_items:0 tracking_total_prefixes:0 unexpected_error_replies:0 total_error_replies:115315 dump_payload_sanitizations:0 total_reads_processed:220146388 total_writes_processed:252455263 io_threaded_reads_processed:158432231 io_threaded_writes_processed:195651021

Replication

role:master connected_slaves:3 slave0:ip=10.137.36.18,port=6379,state=online,offset=93763396009,lag=1 slave1:ip=10.138.36.17,port=6379,state=online,offset=93764848995,lag=0 slave2:ip=10.137.36.19,port=6379,state=online,offset=93763321897,lag=1 master_failover_state:no-failover master_replid:fe351b0e35691dad6f2b756e8f30e4e6453e8c57 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:93764899742 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:93763851167 repl_backlog_histlen:1048576

CPU

used_cpu_sys:3554.247263 used_cpu_user:12517.774097 used_cpu_sys_children:364.659944 used_cpu_user_children:1107.151006 used_cpu_sys_main_thread:2110.901879 used_cpu_user_main_thread:5808.061875

Modules

Errorstats

errorstat_ERR:count=45060 errorstat_LOADING:count=70255

Cluster

cluster_enabled:0

Keyspace

db0:keys=4694,expires=0,avg_ttl=0

1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1420 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"UploadProtocolJob","queue":"default","args":[{"job_class":"UploadProtocolJob","job_id":"c42b165d-f1de-4b6d-865c-3c981254d46e","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"20e8628b-5548-11ed-b5fd-0242eb13228b","activityInstanceId":"uploadProtocolTask:68be2976-60c5-11ed-ae2e-0242ac130002","signalName":"uploadProtocolTask","businessKey":"PKPVDMFC-2022-10-26-236231","region":"66","routeVersion":0,"params":{"status":"pkurp_upload_protocol","_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_3|20e8628b-5548-11ed-b5fd-0242eb13228b/uploadProtocolTask:68be2976-60c5-11ed-ae2e-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-02.int.egrn-46527-1667965026901-31:75:1:1:64547","persistent":"true","region":"66","nodeId":"registration_shard_3","timestamp":"1668063650079","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:17Z"}],"retry":true,"backtrace":true,"jid":"5e825c42b63c03d14f16cf2f","created_at+OK ":1668076577.673476,"enqueued_at":1668076577.9826822} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1407 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"SetPkurpSystemInfoJob","queue":"default","args":[{"job_class":"SetPkurpSystemInfoJob","job_id":"4452894c-e9f5-4cd4-a610-8220b09629b5","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"e3f91ad3-60af-11ed-b9ca-0242ac160002","activityInstanceId":"setPkurpSystemInfoTask:68cc334b-60c5-11ed-b9ca-0242ac160002","signalName":"setPkurpSystemInfoTask","businessKey":"PKPVDMFC-2022-11-10-141613","region":"22","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_4|e3f91ad3-60af-11ed-b9ca-0242ac160002/setPkurpSystemInfoTask:68cc334b-60c5-11ed-b9ca-0242ac160002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-05.int.egrn-39876-1667965175293-31:73:2:1:10595","persistent":"true","region":"22","nodeId":"registration_shard_4","timestamp":"1668063650147","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:17Z"}],"retry":true,"backtrace":true,"jid":"4b0bdb580bb764630678c9ef","created_at":1668076577.+OK 9868073,"enqueued_at":1668076577.997815} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1440 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionJob","queue":"default","args":[{"job_class":"GenerateExtractionJob","job_id":"3549eb63-2208-4f60-a0f2-36833d47a483","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"d00f04c4-6087-11ed-b1f5-0242ac130002","activityInstanceId":"generateExtractionTask:68c3f5a4-60c5-11ed-b1f5-0242ac130002","signalName":"generateExtractionTask","businessKey":"Vedomstvo-2022-11-09-637486","region":"45","routeVersion":0,"params":{"status":"generate_information","_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_2|d00f04c4-6087-11ed-b1f5-0242ac130002/generateExtractionTask:68c3f5a4-60c5-11ed-b1f5-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-01.int.egrn-37852-1667964965833-31:60:1:1:64628","persistent":"true","region":"45","nodeId":"registration_shard_2","timestamp":"1668063650157","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:18Z"}],"retry":true,"backtrace":true,"jid":"907bb08df4d1ba01e+OK 27e5fea","created_at":1668076578.001742,"enqueued_at":1668076578.0963836} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1440 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionJob","queue":"default","args":[{"job_class":"GenerateExtractionJob","job_id":"ef558cc7-6a4f-4263-aa4b-4366a30c23f6","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"98c78fdf-6087-11ed-b1f5-0242ac130002","activityInstanceId":"generateExtractionTask:68c3f5a5-60c5-11ed-b1f5-0242ac130002","signalName":"generateExtractionTask","businessKey":"Vedomstvo-2022-11-09-669880","region":"36","routeVersion":0,"params":{"status":"generate_information","_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_2|98c78fdf-6087-11ed-b1f5-0242ac130002/generateExtractionTask:68c3f5a5-60c5-11ed-b1f5-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-01.int.egrn-37852-1667964965833-31:61:1:1:64628","persistent":"true","region":"36","nodeId":"registration_shard_2","timestamp":"1668063650158","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:18Z"}],"retry":true,"backtrace":true,"jid":"b0db51b9e828fb62a7c4f214","created_at":1668076578.100888,"enqueued_at":1668076578.2489877} 1 $4 exec +OK +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $5 STAMP 3 $5 lpush $11 queue:STAMP $1414 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionStampJob","queue":"STAMP","args":[{"job_class":"GenerateExtractionStampJob","job_id":"cb7a9a93-1833-4947-a840-2b8170fc12e1","provider_job_id":null,"queue_name":"STAMP","priority":null,"arguments":[{"error":null,"executionId":"cf559bd5-5fef-11ed-99a9-0242ac130002","activityInstanceId":"generateExtractionStampTask:68cde12f-60c5-11ed-99a9-0242ac130002","signalName":"generateExtractionStampTask","businessKey":"Vedomstvo-2022-11-09-125821","region":"00","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration|cf559bd5-5fef-11ed-99a9-0242ac130002/generateExtractionStampTask:68cde12f-60c5-11ed-99a9-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-05.int.egrn-39876-1667965175293-31:74:1:1:64537","persistent":"true","region":"00","nodeId":"registration","timestamp":"1668063650165","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:18Z"}],"retry":true,"backtrace":true,"jid":"3652257c868a1bfa1bc0f959","created_at":1668076578.2542686,"enqueued_at":1668076578.2981212} 1 $4 exec +OK +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1414 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"DetermineExtractionTypeJob","queue":"default","args":[{"job_class":"DetermineExtractionTypeJob","job_id":"a36320a9-7034-42a3-a5c8-14ad6df23871","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"68bffe2f-60c5-11ed-99a9-0242ac130002","activityInstanceId":"determineExtractionTypeTask:68cf19b0-60c5-11ed-99a9-0242ac130002","signalName":"determineExtractionTypeTask","businessKey":"PKPVD-2022-11-09-579009","region":"00","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration|68bffe2f-60c5-11ed-99a9-0242ac130002/determineExtractionTypeTask:68cf19b0-60c5-11ed-99a9-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-04.int.egrn-34062-1667965080348-31:55:1:1:64728","persistent":"true","region":"00","nodeId":"registration","timestamp":"1668063650169","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:18Z"}],"retry":true,"backtrace":true,"jid":"ed4bc1d0133c3f346e325af8","created_at":16680+OK 76578.3014197,"enqueued_at":1668076578.3182492} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1441 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionJob","queue":"default","args":[{"job_class":"GenerateExtractionJob","job_id":"712fc663-1fa5-4ea7-85e3-d1c58e743e10","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"fa6c9eed-6086-11ed-ae2e-0242ac130002","activityInstanceId":"generateExtractionTask:68c0737f-60c5-11ed-ae2e-0242ac130002","signalName":"generateExtractionTask","businessKey":"Vedomstvo-2022-11-09-762808","region":"77","routeVersion":0,"params":{"status":"generate_information","_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_3|fa6c9eed-6086-11ed-ae2e-0242ac130002/generateExtractionTask:68c0737f-60c5-11ed-ae2e-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-03.int.egrn-36481-1667965052870-31:64:1:1:64720","persistent":"true","region":"77","nodeId":"registration_shard_3","timestamp":"1668063650179","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:18Z"}],"retry":true,"backtrace":true,"jid":"d4fdd556247146798+OK 767418e","created_at":1668076578.3224347,"enqueued_at":1668076579.9769359} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1390 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionPrefetchSaJob","queue":"default","args":[{"job_class":"GenerateExtractionPrefetchSaJob","job_id":"3693ced9-e24d-4d43-a99e-abb18e9d9851","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"e13d0812-60af-11ed-b9ca-0242ac160002","activityInstanceId":"68d1157a-60c5-11ed-b9ca-0242ac160002","signalName":"generateExtractionPrefetchSaTask","businessKey":"OfSite-2022-11-08-149467","region":"19","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_4|e13d0812-60af-11ed-b9ca-0242ac160002/68d1157a-60c5-11ed-b9ca-0242ac160002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-05.int.egrn-39876-1667965175293-31:47:2:1:64537","persistent":"true","region":"19","nodeId":"registration_shard_4","timestamp":"1668063650182","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:19Z"}],"retry":true,"backtrace":true,"jid":"83792fba48fd7baf5d193159","created_at":1668076579.9810367,"enqueued_at":1668076580.0298808} 1 $4 exec +OK +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1391 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionPrefetchSaJob","queue":"default","args":[{"job_class":"GenerateExtractionPrefetchSaJob","job_id":"d6378ddc-62d2-4447-9e06-a6908acb7f35","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"905ce12a-60b0-11ed-9a6c-0242ac130002","activityInstanceId":"68ce0782-60c5-11ed-9a6c-0242ac130002","signalName":"generateExtractionPrefetchSaTask","businessKey":"OfSite-2022-11-09-1083837","region":"64","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_1|905ce12a-60b0-11ed-9a6c-0242ac130002/68ce0782-60c5-11ed-9a6c-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:70:2:1:64523","persistent":"true","region":"64","nodeId":"registration_shard_1","timestamp":"1668063650182","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:20Z"}],"retry":true,"backtrace":true,"jid":"a3fda04e851e5d518704fe73","created_at":1668076580.0336363,"enqueued_at":1668076580.0982878} 1 $4 exec +OK +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1438 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionJob","queue":"default","args":[{"job_class":"GenerateExtractionJob","job_id":"bd8c3c1f-8e37-4b72-b0e1-5ae8eb3bc665","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"f85b0fbe-6079-11ed-b9ca-0242ac160002","activityInstanceId":"generateExtractionTask:68caaca6-60c5-11ed-b9ca-0242ac160002","signalName":"generateExtractionTask","businessKey":"OfSite-2022-11-09-674920","region":"62","routeVersion":0,"params":{"status":"generate_information","_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_4|f85b0fbe-6079-11ed-b9ca-0242ac160002/generateExtractionTask:68caaca6-60c5-11ed-b9ca-0242ac160002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-05.int.egrn-39876-1667965175293-31:53:2:1:64537","persistent":"true","region":"62","nodeId":"registration_shard_4","timestamp":"1668063650210","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:20Z"}],"retry":true,"backtrace":true,"jid":"d15a0df6698316e108bd7bb0","created_at":1668076580.1022253,"enqueued_at":1668076580.1145947} 1 $4 exec +OK +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1441 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionJob","queue":"default","args":[{"job_class":"GenerateExtractionJob","job_id":"f244f122-639f-431d-be4c-2e666737a80e","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"6151546c-6085-11ed-ae2e-0242ac130002","activityInstanceId":"generateExtractionTask:68d4975e-60c5-11ed-ae2e-0242ac130002","signalName":"generateExtractionTask","businessKey":"Vedomstvo-2022-11-09-650555","region":"66","routeVersion":0,"params":{"status":"generate_information","_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_3|6151546c-6085-11ed-ae2e-0242ac130002/generateExtractionTask:68d4975e-60c5-11ed-ae2e-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-03.int.egrn-36481-1667965052870-31:70:1:1:64720","persistent":"true","region":"66","nodeId":"registration_shard_3","timestamp":"1668063650217","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:20Z"}],"retry":true,"backtrace":true,"jid":"6b1ca03bd0cb4af87+OK d6e6594","created_at":1668076580.1189332,"enqueued_at":1668076580.1768336} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1405 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"SetPkurpSystemInfoJob","queue":"default","args":[{"job_class":"SetPkurpSystemInfoJob","job_id":"b9c01cfc-1c91-4385-85fe-66a27d898a03","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"e4f3ceae-60af-11ed-b9ca-0242ac160002","activityInstanceId":"setPkurpSystemInfoTask:68d693c8-60c5-11ed-b9ca-0242ac160002","signalName":"setPkurpSystemInfoTask","businessKey":"OfSite-2022-11-10-142405","region":"75","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_4|e4f3ceae-60af-11ed-b9ca-0242ac160002/setPkurpSystemInfoTask:68d693c8-60c5-11ed-b9ca-0242ac160002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:50:2:1:64737","persistent":"true","region":"75","nodeId":"registration_shard_4","timestamp":"1668063650220","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:20Z"}],"retry":true,"backtrace":true,"jid":"75ed11b310c078a95df86cd5","created_at":1668076580.1803586,"enqueued_at":1668076580.199957} 1 $4 exec +OK +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1440 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionJob","queue":"default","args":[{"job_class":"GenerateExtractionJob","job_id":"d6fa3274-e74f-4890-9721-9ed0c98a5196","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"fc5079f1-6086-11ed-ae2e-0242ac130002","activityInstanceId":"generateExtractionTask:6895b9f4-60c5-11ed-ae2e-0242ac130002","signalName":"generateExtractionTask","businessKey":"Vedomstvo-2022-11-09-647703","region":"24","routeVersion":0,"params":{"status":"generate_information","_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_3|fc5079f1-6086-11ed-ae2e-0242ac130002/generateExtractionTask:6895b9f4-60c5-11ed-ae2e-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:59:3:1:64689","persistent":"true","region":"24","nodeId":"registration_shard_3","timestamp":"1668063650429","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:20Z"}],"retry":true,"backtrace":true,"jid":"a3a33ddc32f19ec3fc31fbad","created_at":1668076580.204076,"enqueued_at":1668076580.2273088} 1 $4 exec +OK +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1408 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"SetPkurpSystemInfoJob","queue":"default","args":[{"job_class":"SetPkurpSystemInfoJob","job_id":"8faa444a-cfdb-45ad-bf7d-ce7069c8b054","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"199af6f0-60b0-11ed-b9ca-0242ac160002","activityInstanceId":"setPkurpSystemInfoTask:68f89a77-60c5-11ed-b9ca-0242ac160002","signalName":"setPkurpSystemInfoTask","businessKey":"PKPVDMFC-2022-11-10-141726","region":"14","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_4|199af6f0-60b0-11ed-b9ca-0242ac160002/setPkurpSystemInfoTask:68f89a77-60c5-11ed-b9ca-0242ac160002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-04.int.egrn-34062-1667965080348-31:57:1:1:64729","persistent":"true","region":"14","nodeId":"registration_shard_4","timestamp":"1668063650441","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:20Z"}],"retry":true,"backtrace":true,"jid":"f27efec7832cccebf9eb3ea6","created_at":1668076580.+OK 2312381,"enqueued_at":1668076580.7355466} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $5 STAMP 3 $5 lpush $11 queue:STAMP $1425 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionStampJob","queue":"STAMP","args":[{"job_class":"GenerateExtractionStampJob","job_id":"fdc13008-3b1f-4e58-a69c-f7068471e5d0","provider_job_id":null,"queue_name":"STAMP","priority":null,"arguments":[{"error":null,"executionId":"180a7578-601d-11ed-a7c9-0242ac130002","activityInstanceId":"generateExtractionStampTask:68fae472-60c5-11ed-b9ca-0242ac160002","signalName":"generateExtractionStampTask","businessKey":"OfSite-2022-11-09-291986","region":"59","routeVersion":0,"params":{"aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_4|180a7578-601d-11ed-a7c9-0242ac130002/generateExtractionStampTask:68fae472-60c5-11ed-b9ca-0242ac160002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:65:1:1:64817","persistent":"true","region":"59","nodeId":"registration_shard_4","timestamp":"1668063650457","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:20Z"}],"retry":true,"backtrace":true,"jid":"92e335e00420fbefa7350b80","created+OK at":1668076580.74111,"enqueued_at":1668076581.1355124} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1416 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"CreatePreliminaryExaminationProtocolJob","queue":"default","args":[{"job_class":"CreatePreliminaryExaminationProtocolJob","job_id":"752a10e0-e39c-443a-a003-724139759c95","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"025637ab-609d-11ed-b1f5-0242ac130002","activityInstanceId":"68f9ac03-60c5-11ed-b1f5-0242ac130002","signalName":"createPreliminaryExaminationProtocolTask","businessKey":"PKPVDMFC-2022-11-09-470191","region":"37","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_2|025637ab-609d-11ed-b1f5-0242ac130002/68f9ac03-60c5-11ed-b1f5-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-04.int.egrn-34062-1667965080348-31:59:1:1:64729","persistent":"true","region":"37","nodeId":"registration_shard_2","timestamp":"1668063650474","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:21Z"}],"retry":true,"backtrace":true,"jid":"f7a4dd249b40f177f6ce7fc2","created_at":166+OK 8076581.1386635,"enqueued_at":1668076581.1955469} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1404 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"SetPkurpSystemInfoJob","queue":"default","args":[{"job_class":"SetPkurpSystemInfoJob","job_id":"e8728a27-2195-4b6c-b6b1-8287c950eb46","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"7936f012-60af-11ed-b1f5-0242ac130002","activityInstanceId":"setPkurpSystemInfoTask:6951b704-60c5-11ed-b1f5-0242ac130002","signalName":"setPkurpSystemInfoTask","businessKey":"PKPVD-2022-11-10-140480","region":"63","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_2|7936f012-60af-11ed-b1f5-0242ac130002/setPkurpSystemInfoTask:6951b704-60c5-11ed-b1f5-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-02.int.egrn-46527-1667965026901-31:67:1:1:64550","persistent":"true","region":"63","nodeId":"registration_shard_2","timestamp":"1668063651048","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:21Z"}],"retry":true,"backtrace":true,"jid":"fe408f03a3fcfe436d551dbe","created_at":1668076581.199+OK 2948,"enqueued_at":1668076581.224526} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1401 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"SetPkurpSystemInfoJob","queue":"default","args":[{"job_class":"SetPkurpSystemInfoJob","job_id":"7bb3a3ce-0899-4960-ba9b-126d8786c68c","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"04e1c64a-60ac-11ed-ae2e-0242ac130002","activityInstanceId":"setPkurpSystemInfoTask:69d258eb-60c5-11ed-ae2e-0242ac130002","signalName":"setPkurpSystemInfoTask","businessKey":"BC-2022-11-10-128553","region":"24","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_3|04e1c64a-60ac-11ed-ae2e-0242ac130002/setPkurpSystemInfoTask:69d258eb-60c5-11ed-ae2e-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:69:1:1:64822","persistent":"true","region":"24","nodeId":"registration_shard_3","timestamp":"1668063651868","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:21Z"}],"retry":true,"backtrace":true,"jid":"53a6ae3070ec0ba6d035907a","created_at":1668076581.228157,"enqueued_at":1668076581.2535846} 1 $4 exec +OK +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1441 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionJob","queue":"default","args":[{"job_class":"GenerateExtractionJob","job_id":"790f7f7e-3cd9-4519-a6e5-f6550c8b0201","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"140b9ca6-6087-11ed-ae2e-0242ac130002","activityInstanceId":"generateExtractionTask:6989b6f2-60c5-11ed-ae2e-0242ac130002","signalName":"generateExtractionTask","businessKey":"Vedomstvo-2022-11-09-647681","region":"24","routeVersion":0,"params":{"status":"generate_information","_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_3|140b9ca6-6087-11ed-ae2e-0242ac130002/generateExtractionTask:6989b6f2-60c5-11ed-ae2e-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:70:2:1:64530","persistent":"true","region":"24","nodeId":"registration_shard_3","timestamp":"1668063651870","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:21Z"}],"retry":true,"backtrace":true,"jid":"97490aade79a29b43+OK 6e746f3","created_at":1668076581.2580307,"enqueued_at":1668076581.3376317} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1400 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"CreateClaimsJob","queue":"default","args":[{"job_class":"CreateClaimsJob","job_id":"6ad45582-28fc-4bec-abe5-3e57cb901be1","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"85cad856-60af-11ed-b9ca-0242ac160002","activityInstanceId":"createClaimsTask:69d42d1b-60c5-11ed-b9ca-0242ac160002","signalName":"createClaimsTask","businessKey":"PKPVDMFC-2022-11-10-100909","region":"14","routeVersion":0,"params":{"status":"valueStatus","_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_4|85cad856-60af-11ed-b9ca-0242ac160002/createClaimsTask:69d42d1b-60c5-11ed-b9ca-0242ac160002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-02.int.egrn-46527-1667965026901-31:51:1:1:64555","persistent":"true","region":"14","nodeId":"registration_shard_4","timestamp":"1668063651879","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:21Z"}],"retry":true,"backtrace":true,"jid":"e06dda0795f5d312c65fbf01","created_at":1668076581.3417642+OK ,"enqueued_at":1668076581.519074} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1388 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionPrefetchSaJob","queue":"default","args":[{"job_class":"GenerateExtractionPrefetchSaJob","job_id":"7ca5e03d-d327-4703-a653-ed2bf810686b","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"ca2d8fa4-60af-11ed-b9ca-0242ac160002","activityInstanceId":"69d4f070-60c5-11ed-b9ca-0242ac160002","signalName":"generateExtractionPrefetchSaTask","businessKey":"OfSite-2022-11-08-311345","region":"59","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_4|ca2d8fa4-60af-11ed-b9ca-0242ac160002/69d4f070-60c5-11ed-b9ca-0242ac160002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:73:1:1:64822","persistent":"true","region":"59","nodeId":"registration_shard_4","timestamp":"1668063651883","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:21Z"}],"retry":true,"backtrace":true,"jid":"43702fc8f1e60e0e85bbb152","created_at":1668076581.522981,"enqueued_a+OK t":1668076581.705489} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1389 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionPrefetchSaJob","queue":"default","args":[{"job_class":"GenerateExtractionPrefetchSaJob","job_id":"697255b5-0e84-422a-bd82-69de6626c40e","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"1c577e75-60b0-11ed-b9ca-0242ac160002","activityInstanceId":"69d4c95f-60c5-11ed-b9ca-0242ac160002","signalName":"generateExtractionPrefetchSaTask","businessKey":"Other-2022-11-09-574970","region":"34","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_4|1c577e75-60b0-11ed-b9ca-0242ac160002/69d4c95f-60c5-11ed-b9ca-0242ac160002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:74:1:1:64822","persistent":"true","region":"34","nodeId":"registration_shard_4","timestamp":"1668063651885","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:21Z"}],"retry":true,"backtrace":true,"jid":"5aa492a5e19bf3cdae70982b","created_at":1668076581.7094886,"enqueued_a+OK t":1668076581.7732441} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $5 STAMP 3 $5 lpush $11 queue:STAMP $1407 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionStampJob","queue":"STAMP","args":[{"job_class":"GenerateExtractionStampJob","job_id":"ec400309-5762-49b5-9c20-6f92b6d569e8","provider_job_id":null,"queue_name":"STAMP","priority":null,"arguments":[{"error":null,"executionId":"320b09a7-5fef-11ed-99a9-0242ac130002","activityInstanceId":"generateExtractionStampTask:69d3df93-60c5-11ed-99a9-0242ac130002","signalName":"generateExtractionStampTask","businessKey":"US-2022-11-09-127200","region":"00","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration|320b09a7-5fef-11ed-99a9-0242ac130002/generateExtractionStampTask:69d3df93-60c5-11ed-99a9-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-02.int.egrn-46527-1667965026901-31:54:3:1:64553","persistent":"true","region":"00","nodeId":"registration","timestamp":"1668063651896","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:21Z"}],"retry":true,"backtrace":true,"jid":"d50c4b144e872ef721fb9643","created_at":1668076581.7772045,"enqueued_at":1668076581.8426898} 1 $4 exec +OK +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1441 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionJob","queue":"default","args":[{"job_class":"GenerateExtractionJob","job_id":"868a71cd-ddbe-4c11-aba0-a61a83bb82fc","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"f3a7606c-6086-11ed-ae2e-0242ac130002","activityInstanceId":"generateExtractionTask:6969abfd-60c5-11ed-ae2e-0242ac130002","signalName":"generateExtractionTask","businessKey":"Vedomstvo-2022-11-09-690843","region":"24","routeVersion":0,"params":{"status":"generate_information","_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_3|f3a7606c-6086-11ed-ae2e-0242ac130002/generateExtractionTask:6969abfd-60c5-11ed-ae2e-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:54:10:1:10691","persistent":"true","region":"24","nodeId":"registration_shard_3","timestamp":"1668063651916","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:21Z"}],"retry":true,"backtrace":true,"jid":"db008808d2188019379bfab0","created_at":1668076581.846275,"enqueued_at":1668076581.8976755} 1 $4 exec +OK +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1407 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"SetPkurpSystemInfoJob","queue":"default","args":[{"job_class":"SetPkurpSystemInfoJob","job_id":"c53c87ab-03b1-48db-b2ac-9f58dab328f3","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"ffa8d7b1-60ac-11ed-ae2e-0242ac130002","activityInstanceId":"setPkurpSystemInfoTask:69da965e-60c5-11ed-ae2e-0242ac130002","signalName":"setPkurpSystemInfoTask","businessKey":"Vedomstvo-2022-11-10-133198","region":"77","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_3|ffa8d7b1-60ac-11ed-ae2e-0242ac130002/setPkurpSystemInfoTask:69da965e-60c5-11ed-ae2e-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:57:1:1:10691","persistent":"true","region":"77","nodeId":"registration_shard_3","timestamp":"1668063651928","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:21Z"}],"retry":true,"backtrace":true,"jid":"8d1e501d08763c20b6656054","created_at":1668076581+OK .9022496,"enqueued_at":1668076581.94914} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1401 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"SetPkurpSystemInfoJob","queue":"default","args":[{"job_class":"SetPkurpSystemInfoJob","job_id":"f4de7c2c-4b32-4b2c-8e04-c12b714aa32d","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"501ba138-60ad-11ed-ae2e-0242ac130002","activityInstanceId":"setPkurpSystemInfoTask:69d936cb-60c5-11ed-ae2e-0242ac130002","signalName":"setPkurpSystemInfoTask","businessKey":"BC-2022-11-10-134783","region":"66","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_3|501ba138-60ad-11ed-ae2e-0242ac130002/setPkurpSystemInfoTask:69d936cb-60c5-11ed-ae2e-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:58:1:1:10636","persistent":"true","region":"66","nodeId":"registration_shard_3","timestamp":"1668063651928","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:21Z"}],"retry":true,"backtrace":true,"jid":"d821006c2ccb6dc5a8e0cdb2","created_at":1668076581.952651,"enqueued_at":1668076581.9700603} 1 $4 exec +OK +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1408 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"SetPkurpSystemInfoJob","queue":"default","args":[{"job_class":"SetPkurpSystemInfoJob","job_id":"3d62a7e6-d05a-427c-a801-ac9e9922e57f","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"220e8911-60ad-11ed-ae2e-0242ac130002","activityInstanceId":"setPkurpSystemInfoTask:69db32a0-60c5-11ed-ae2e-0242ac130002","signalName":"setPkurpSystemInfoTask","businessKey":"Vedomstvo-2022-11-10-133375","region":"77","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_3|220e8911-60ad-11ed-ae2e-0242ac130002/setPkurpSystemInfoTask:69db32a0-60c5-11ed-ae2e-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:60:2:1:64745","persistent":"true","region":"77","nodeId":"registration_shard_3","timestamp":"1668063651937","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:21Z"}],"retry":true,"backtrace":true,"jid":"ac70ec8507200c81ce9c26bc","created_at":1668076581+OK .975081,"enqueued_at":1668076581.9905639} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1433 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionJob","queue":"default","args":[{"job_class":"GenerateExtractionJob","job_id":"ea675a45-36cd-44a9-82d6-d29e395b431f","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"f5269057-6087-11ed-b1f5-0242ac130002","activityInstanceId":"generateExtractionTask:69d69ea0-60c5-11ed-b1f5-0242ac130002","signalName":"generateExtractionTask","businessKey":"BC-2022-11-09-658307","region":"74","routeVersion":0,"params":{"status":"generate_information","_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_2|f5269057-6087-11ed-b1f5-0242ac130002/generateExtractionTask:69d69ea0-60c5-11ed-b1f5-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-03.int.egrn-36481-1667965052870-31:47:2:1:10628","persistent":"true","region":"74","nodeId":"registration_shard_2","timestamp":"1668063651940","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:21Z"}],"retry":true,"backtrace":true,"jid":"d3b3c24b07253aa5c1701134+OK ","created_at":1668076581.993959,"enqueued_at":1668076582.0163789} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1441 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionJob","queue":"default","args":[{"job_class":"GenerateExtractionJob","job_id":"f8312b37-1fa8-4aa4-8d14-729b4d68e2c0","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"dac35850-6087-11ed-b1f5-0242ac130002","activityInstanceId":"generateExtractionTask:69db3297-60c5-11ed-b1f5-0242ac130002","signalName":"generateExtractionTask","businessKey":"Vedomstvo-2022-11-09-683012","region":"36","routeVersion":0,"params":{"status":"generate_information","_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_2|dac35850-6087-11ed-b1f5-0242ac130002/generateExtractionTask:69db3297-60c5-11ed-b1f5-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:62:1:1:64822","persistent":"true","region":"36","nodeId":"registration_shard_2","timestamp":"1668063651942","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:22Z"}],"retry":true,"backtrace":true,"jid":"b2937557f65f56f47+OK d322f4e","created_at":1668076582.0200613,"enqueued_at":1668076582.0331862} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1434 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionJob","queue":"default","args":[{"job_class":"GenerateExtractionJob","job_id":"81cfa2f8-2676-4974-952b-91fefb4b3c00","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"0b818618-6088-11ed-b1f5-0242ac130002","activityInstanceId":"generateExtractionTask:69d8e89e-60c5-11ed-b1f5-0242ac130002","signalName":"generateExtractionTask","businessKey":"BC-2022-11-09-690763","region":"72","routeVersion":0,"params":{"status":"generate_information","_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_2|0b818618-6088-11ed-b1f5-0242ac130002/generateExtractionTask:69d8e89e-60c5-11ed-b1f5-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-05.int.egrn-39876-1667965175293-31:53:2:1:64544","persistent":"true","region":"72","nodeId":"registration_shard_2","timestamp":"1668063651942","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:22Z"}],"retry":true,"backtrace":true,"jid":"7603ccc137c66ddcc978e0db+OK ","created_at":1668076582.0382988,"enqueued_at":1668076582.0622268} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1400 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"CreatePreliminaryExaminationProtocolJob","queue":"default","args":[{"job_class":"CreatePreliminaryExaminationProtocolJob","job_id":"a19f0140-2cb5-45fd-bfb5-b5f78dbfdea5","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"971649de-60c4-11ed-99a9-0242ac130002","activityInstanceId":"69dcb938-60c5-11ed-99a9-0242ac130002","signalName":"createPreliminaryExaminationProtocolTask","businessKey":"PKPVDMFC-2022-11-09-925364","region":"00","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration|971649de-60c4-11ed-99a9-0242ac130002/69dcb938-60c5-11ed-99a9-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-02.int.egrn-46527-1667965026901-31:59:1:1:64555","persistent":"true","region":"00","nodeId":"registration","timestamp":"1668063651947","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:22Z"}],"retry":true,"backtrace":true,"jid":"5fca246dc5cd0ecb6a79bb88","created_at":1668076582.0660274,+OK "enqueued_at":1668076582.0834417} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1393 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionPrefetchSaJob","queue":"default","args":[{"job_class":"GenerateExtractionPrefetchSaJob","job_id":"775cee7d-a8ba-42ac-8fd3-91cb1037b726","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"ef87052e-60af-11ed-b9ca-0242ac160002","activityInstanceId":"69df02c2-60c5-11ed-b9ca-0242ac160002","signalName":"generateExtractionPrefetchSaTask","businessKey":"Vedomstvo-2022-11-09-1053236","region":"34","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_4|ef87052e-60af-11ed-b9ca-0242ac160002/69df02c2-60c5-11ed-b9ca-0242ac160002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:46:2:1:10590","persistent":"true","region":"34","nodeId":"registration_shard_4","timestamp":"1668063651949","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:22Z"}],"retry":true,"backtrace":true,"jid":"fd60d991740cb8ad9f104f06","created_at":1668076582.087081,"enqueued_at":1668076582.1055431} 1 $4 exec +OK +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1440 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionJob","queue":"default","args":[{"job_class":"GenerateExtractionJob","job_id":"d0529903-3f2c-482d-a136-2cabef889f85","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"1207dd76-6087-11ed-ae2e-0242ac130002","activityInstanceId":"generateExtractionTask:69cbf020-60c5-11ed-ae2e-0242ac130002","signalName":"generateExtractionTask","businessKey":"Vedomstvo-2022-11-09-689258","region":"77","routeVersion":0,"params":{"status":"generate_information","_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_3|1207dd76-6087-11ed-ae2e-0242ac130002/generateExtractionTask:69cbf020-60c5-11ed-ae2e-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:63:1:1:64823","persistent":"true","region":"77","nodeId":"registration_shard_3","timestamp":"1668063651952","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:22Z"}],"retry":true,"backtrace":true,"jid":"025cfd313ec84d628+OK 0c0838c","created_at":1668076582.1101425,"enqueued_at":1668076582.119382} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1382 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"AcquireRegistryRecordsLockJob","queue":"default","args":[{"job_class":"AcquireRegistryRecordsLockJob","job_id":"4551cd96-22b1-4ac5-b29a-ba44309f6b9d","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"69cf23ef-60c5-11ed-b9ca-0242ac160002","activityInstanceId":"69dfc615-60c5-11ed-b9ca-0242ac160002","signalName":"acquireRegistryRecordsLockTask","businessKey":"Other-2022-11-07-982808","region":"22","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_4|69cf23ef-60c5-11ed-b9ca-0242ac160002/69dfc615-60c5-11ed-b9ca-0242ac160002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:66:1:1:10883","persistent":"true","region":"22","nodeId":"registration_shard_4","timestamp":"1668063651960","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:22Z"}],"retry":true,"backtrace":true,"jid":"2077bf260256a2a710a1531b","created_at":1668076582.124301,"enqueued_at":1668+OK 076610.6074243} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1383 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"AcquireRegistryRecordsLockJob","queue":"default","args":[{"job_class":"AcquireRegistryRecordsLockJob","job_id":"a09b9a33-0eb7-419b-9b89-6d8d1c115570","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"69b0a000-60c5-11ed-9a6c-0242ac130002","activityInstanceId":"69e321ed-60c5-11ed-9a6c-0242ac130002","signalName":"acquireRegistryRecordsLockTask","businessKey":"Other-2022-11-07-649860","region":"54","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_1|69b0a000-60c5-11ed-9a6c-0242ac130002/69e321ed-60c5-11ed-9a6c-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-01.int.egrn-37852-1667964965833-31:52:2:1:10596","persistent":"true","region":"54","nodeId":"registration_shard_1","timestamp":"1668063652012","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:36:50Z"}],"retry":true,"backtrace":true,"jid":"5cc05b8669d18ad3356a1f6e","created_at":1668076610.6127558,"enqueued_at":166+OK 8076632.0953941} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1433 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GenerateExtractionJob","queue":"default","args":[{"job_class":"GenerateExtractionJob","job_id":"38b1425e-54a4-42d5-9ad6-a3243a2f106f","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"fab328be-6087-11ed-b1f5-0242ac130002","activityInstanceId":"generateExtractionTask:69e321e1-60c5-11ed-b1f5-0242ac130002","signalName":"generateExtractionTask","businessKey":"BC-2022-11-09-708108","region":"74","routeVersion":0,"params":{"status":"generate_information","_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_2|fab328be-6087-11ed-b1f5-0242ac130002/generateExtractionTask:69e321e1-60c5-11ed-b1f5-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:74:1:1:64823","persistent":"true","region":"74","nodeId":"registration_shard_2","timestamp":"1668063652016","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:37:12Z"}],"retry":true,"backtrace":true,"jid":"fb4e85acb68e838496a963c6+OK ","created_at":1668076632.100281,"enqueued_at":1668076632.1225903} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $23 FetchRegistryRecordsJob 3 $5 lpush $29 queue:FetchRegistryRecordsJob $1447 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"FetchRegistryRecordsJob","queue":"FetchRegistryRecordsJob","args":[{"job_class":"FetchRegistryRecordsJob","job_id":"bc275041-19d8-4c02-a2ee-3e2dca172233","provider_job_id":null,"queue_name":"FetchRegistryRecordsJob","priority":null,"arguments":[{"error":null,"executionId":"fd66afcf-6087-11ed-b9ca-0242ac160002","activityInstanceId":"fetchRegistryRecordsTask:6a6c2771-60c5-11ed-b9ca-0242ac160002","signalName":"fetchRegistryRecordsTask","businessKey":"Other-2022-11-09-1129844","region":"34","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_4|fd66afcf-6087-11ed-b9ca-0242ac160002/fetchRegistryRecordsTask:6a6c2771-60c5-11ed-b9ca-0242ac160002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-04.int.egrn-34062-1667965080348-31:65:2:1:10603","persistent":"true","region":"34","nodeId":"registration_shard_4","timestamp":"1668063652874","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:37:12Z"}],"retry":true,"backtrace":true,"jid":"9c75299fd0542ec2620408b2","created_at":1668076632.1264946,"enqueued_at":1668076632.164624} 1 $4 exec +OK +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $23 FetchRegistryRecordsJob 3 $5 lpush $29 queue:FetchRegistryRecordsJob $1448 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"FetchRegistryRecordsJob","queue":"FetchRegistryRecordsJob","args":[{"job_class":"FetchRegistryRecordsJob","job_id":"7ce80fe5-6214-4289-9f6a-47bb720c690d","provider_job_id":null,"queue_name":"FetchRegistryRecordsJob","priority":null,"arguments":[{"error":null,"executionId":"fea6d09c-6087-11ed-b9ca-0242ac160002","activityInstanceId":"fetchRegistryRecordsTask:6a6c7593-60c5-11ed-b9ca-0242ac160002","signalName":"fetchRegistryRecordsTask","businessKey":"Other-2022-11-09-1129405","region":"34","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_4|fea6d09c-6087-11ed-b9ca-0242ac160002/fetchRegistryRecordsTask:6a6c7593-60c5-11ed-b9ca-0242ac160002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-06.int.egrn-44036-1667965237948-31:55:3:1:10591","persistent":"true","region":"34","nodeId":"registration_shard_4","timestamp":"1668063652877","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:37:12Z"}],"retry":true,"+OK backtrace":true,"jid":"3a1bde1d9dd3d35595ec10cc","created_at":1668076632.1686008,"enqueued_at":1668076632.1777244} 1 $4 exec +QUEUED +QUEUED 2 :0 :1 1 $5 multi 3 $4 sadd $6 queues $7 default 3 $5 lpush $13 queue:default $1407 {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"SetPkurpSystemInfoJob","queue":"default","args":[{"job_class":"SetPkurpSystemInfoJob","job_id":"e25109fc-9335-4027-8a59-2c9f6c8eaa54","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"error":null,"executionId":"06363daa-60ad-11ed-ae2e-0242ac130002","activityInstanceId":"setPkurpSystemInfoTask:6a6cc36c-60c5-11ed-ae2e-0242ac130002","signalName":"setPkurpSystemInfoTask","businessKey":"PKPVDMFC-2022-11-10-131404","region":"66","routeVersion":0,"params":{"_aj_symbol_keys":[]},"_aj_symbol_keys":[]},{"expires":"0","correlation-id":"PKURP:registration_shard_3|06363daa-60ad-11ed-ae2e-0242ac130002/setPkurpSystemInfoTask:6a6cc36c-60c5-11ed-ae2e-0242ac130002","destination":"/queue/pkurpIn","CamelJmsDeliveryMode":"2","subscription":"ad801ea4236c8916ce3469b5d96310e990b09ba8","priority":"4","command":"ru.progress.rosreestr.pkurp.cmd.SimpleSignalProcessCommand","message-id":"ID:dc1-ppoz-service-external-02.int.egrn-46527-1667965026901-31:62:1:1:64559","persistent":"true","region":"66","nodeId":"registration_shard_3","timestamp":"1668063652879","_aj_symbol_keys":[]}],"executions":0,"exception_executions":{},"locale":"ru","timezone":"UTC","enqueued_at":"2022-11-10T10:37:12Z"}],"retry":true,"backtrace":true,"jid":"73e3a59c33d32b4464ecbc81","created_at":1668076632.+OK 181309,"enqueued_at":1668076632.2398095} 1 $4 exec +QUE

Comment From: olegredis

info.txt redis_pull_messages.txt Hi Please see the attached files