Implementing a buffer structure in Python

I am trying to write a small wsgi application that will put some objects into an external queue after every request. I want to do it in batch, i.e. make the web server put the object in a buffer-like structure in memory, and another thread and / or process to send those objects to the queue in batch mode when the buffer is large enough or after a certain timeout, and flush the buffer. I don’t want to be in the NIH syndrome and don’t want to worry about streaming material, however I couldn’t find the right code to make it work. Any suggestions?

+1


source to share


3 answers


Check out https://docs.python.org/library/queue.html to see if it suits your needs.



+7


source


Since you are writing "thread and / or process" see also multiprocessing. Queue and multiprocessing.JoinableQueue from 2.6. These are interprocess queue options.



+2


source


Use a buffered stream if you are using python 3.0.

+1


source







All Articles