Hey guys! I start using Redis recently so I dont get it right. Can you help me please?

I face error Error while reading from localhost:6379: (10054, 'The remote host forcibly terminated an existing connection', None, 10054, None (translated from Russian)).

What do I do? I have a file named redis_client in which I have this code:

import redis

connection_pool = redis.ConnectionPool(host='localhost', port=6379, db=0)

def set_value(key, value):
    with redis.Redis(connection_pool=connection_pool) as redis_client:
        redis_client.set(key, value)

def get_value(key):
    with redis.Redis(connection_pool=connection_pool) as redis_client:
        return redis_client.get(key)

Then I have two files which do from redis_client import set_value, get_value and use set and get methods in their work

And this is it. These two files work at the same time, parallel, and with a very high frequency (one file get data from websocket, than get data from Redis, change it and set it back in Redis updated, while second file get updated data from Redis and analyse it). My second file work for a minute-two and then got this error: Error while reading from localhost:6379: (10054, 'The remote host forcibly terminated an existing connection', None, 10054, None

So why does it happen and how to fix it?

Comment From: alexgershberg

Would you mind sharing the entire code? (maybe place code from both files together, which consumes get_value and set_value)

also, an exact screenshot of the error would be great (with a call stack, if there is one?)

I’m not terribly familiar with redis, but maybe the log from the redis-server would be useful too

Comment From: KindSpidey

Would you mind sharing the entire code? (maybe place code from both files together, which consumes get_value and set_value)

also, an exact screenshot of the error would be great (with a call stack, if there is one?)

I’m not terribly familiar with redis, but maybe the log from the redis-server would be useful too

Sure!

Lets step by step:

1)I have redis_client file, where I have pool and two methods, which I use when working with Redis:

import redis

connection_pool = redis.ConnectionPool(host='localhost', port=6379, db=0)

def set_value(key, value):
    with redis.Redis(connection_pool=connection_pool) as redis_client:
        redis_client.set(key, value)

def get_value(key):
    with redis.Redis(connection_pool=connection_pool) as redis_client:
        return redis_client.get(key)

2) I have my websocket file, where I read orderbook data and set it into Redis. this is how I do it:

parsed_message = json.loads(message)
id = parsed_message['arg']['instId']
asks = parsed_message['data'][0]['asks']
bids = parsed_message['data'][0]['bids']
value_from_redis = get_value(str(exchange) + '_' + id)
existing_data = json.loads(value_from_redis)
existing_data['asks'] = asks
existing_data['bids'] = bids
updated_json_value = json.dumps(existing_data)
set_value(str(exchange) + '_' + id, updated_json_value)

I do this for 600+ times at the same time (600 channels to listen)

3) I have analyser file, where, in while True, for ~100 000 times at the same time I do this:

data = json.loads(get_value(redis_id)) just getting data, not setting anything

Error I got is - (Error while reading from localhost:6379: (10054, 'Удаленный хост принудительно разорвал существующее подключение', None, 10054, None)

logs from redis server just saying:

[1104] 14 Aug 11:52:37.078 # Server started, Redis version 3.0.504
[1104] 14 Aug 11:52:37.122 * DB loaded from disk: 0.044 seconds
[1104] 14 Aug 11:52:37.122 * The server is now ready to accept connections on port 6379
[1104] 14 Aug 11:53:38.041 * 10000 changes in 60 seconds. Saving...
[1104] 14 Aug 11:53:38.045 * Background saving started by pid 16264
[1104] 14 Aug 11:53:38.445 # fork operation complete
[1104] 14 Aug 11:53:38.453 * Background saving terminated with success
[1104] 14 Aug 11:54:39.024 * 10000 changes in 60 seconds. Saving...
[1104] 14 Aug 11:54:39.030 * Background saving started by pid 97596
[1104] 14 Aug 11:54:40.150 # fork operation complete
[1104] 14 Aug 11:54:40.197 * Background saving terminated with success
[1104] 14 Aug 11:55:41.007 * 10000 changes in 60 seconds. Saving...
[1104] 14 Aug 11:55:41.010 * Background saving started by pid 4292
[1104] 14 Aug 11:55:41.229 # fork operation complete
[1104] 14 Aug 11:55:41.233 * Background saving terminated with success

Comment From: alexgershberg

Thank you! I’ll try to reproduce and take a closer look :)

Comment From: sundb

@KindSpidey If the logs is complete, we can see that the connections weren't disconnected by Redis. Maybe you can use config LOGLEVEL DEBUG to see if there are any valuable informations.