python - redis-py subscribe blocked when read message -
recentlly use python , redis build smarl messge-driven project. use 1 thread subsribe redis channel(called message thread here); timer thread; , worker thread; when message thread got enough messages, post task worker. use redis-py communicate redis
message thread:
subscribe redis;
while true: message; if len(messages)>threashold: post task worker
worker thread:
while true: wait task event; task; //this may heavy
here comes problem: after work while, redis-py subpub blocked!(ofcource redis still publish message, not return anymore, blocked!). use gdb attach it, see stack frame this:
[switching thread 4 (thread 1084229984 (lwp 9812))]#0 0x000000302b80b0cf in __read_nocancel () /lib64/tls/libpthread.so.0 (gdb) bt 0 0x000000302b80b0cf in __read_nocancel () /lib64/tls/libpthread.so.0
1 0x00000000004e129a in posix_read (self=variable "self" not available.) at./modules/posixmodule.c:6592
2 0x00000000004a04c5 in pyeval_evalframeex (f=0x157a8c0, throwflag=variable "throwflag" not available.) @ python/ceval.c:4323
i use redis 'client kill' command kill connection between python , redis, python still block there, never return or raise exeption. way kill python process use kill -9.
then comment work's 'do task' procedure(remember task heavy, make heavy network io, cpu calculation), works , no problem abserved.
so, seems come conclusion: once use worker task, message thread block @ socket read.
how can happen!!
the probable explanation use same redis connection in task processing code. should not.
once connection has subscribed, cannot use except receiving messages, or running additional subscribe, psubscribe, unsubscribe , punsubscribe commands.
you need second redis connection in task processing code.
Comments
Post a Comment