Python streams freeze and not close
This is my first attempt at streaming in Python.I wrote the following program as a very simple example. It just gets the list and prints it with some streams. However, whenever there is an error, the program just hangs in Ubuntu and I can't do anything to get the control prompt back, so reboot another SSH session to get back. Also don't know what is the problem with my program. Is there some kind of error handling that I can enable so that it doesn't hang. Also, any idea why ctrl / c isn't working (I don't have a break key)
from Queue import Queue
from threading import Thread
import HAInstances
import logging
log = logging.getLogger()
logging.basicConfig()
class GetHAInstances:
def oraHAInstanceData(self):
log.info('Getting HA instance routing data')
# HAData = SolrGetHAInstances.TalkToOracle.main()
HAData = HAInstances.main()
log.info('Query fetched ' + str(len(HAData)) + ' HA Instances to query')
# for row in HAData:
# print row
return(HAData)
def do_stuff(q):
while True:
print q.get()
print threading.current_thread().name
q.task_done()
oraHAInstances = GetHAInstances()
mainHAData = oraHAInstances.oraHAInstanceData()
q = Queue(maxsize=0)
num_threads = 10
for i in range(num_threads):
worker = Thread(target=do_stuff, args=(q,))
worker.setDaemon(True)
worker.start()
for row in mainHAData:
#print str(row[0]) + ':' + str(row[1]) + ':' + str(row[2]) + ':' + str(row[3])i
q.put((row[0],row[1],row[2],row[3]))
q.join()
source to share
It is recommended to use "try ... except ... finally" in a stream method. This structure ensures that the control returns to the main thread even if errors occur.
def do_stuff(q):
while True:
try:
#do your works
except:
#log the error
finally:
q.task_done()
Also, if you want to kill your program, go find the pid of your main thread and use kill #pid
it to kill it. On Ubuntu or Mint use ps -Ao pid,cmd
, in the output you can find out the pid (first column) by searching for the command (second column) you typed yourself to run the Python script.
source to share