Why does the io.BytesIO stream close when passed as an argument?
I have a Django app where the user is uploading a file. The view grabs the file, passes it to the stream function, and processes it. The problem is that the io.BytesIO stream is closed before it can be read, and I cannot figure out why it closes.
I am processing Excel files using the Xlrd library. For some Excel downloads, the stream remains open, but some are close. I don't understand what is going on under the hood to understand why. What I'm asking is debug advice. I cannot provide my complete code, but I have outlined what the code looks like below.
My Django view grabs the file from the POST request:
ufile = request.FILES['upload-file'].file
ufile
is an instance io.BytesIO
. I have a function that copies a stream to a named temporary file
@contextmanager
def temp_input_file(file_):
temp = tempfile.NamedTemporaryFile(delete=False)
shutil.copyfileobj(file_, temp)
temp.close()
yield temp.name
os.unlink(temp.name)
This decoration function of the contextmanager is used a bit later. The next step is to create a stream to process the uploaded file:
job.run_job(
method=process_excel_file,
uploaded_file=ufile
)
What I am doing is pass the function process_excel_file
and the uploaded "file", but actually the stream io.BytesIO
to job.run_job that creates the stream
def run_job(self, method, **kwargs):
thread = threading.Thread()
thread.run = lambda : method(
job_id=self.job_db_record.id,
**kwargs
)
thread.start()
So, it process_excel_file
is passed to a variable method
and the ufile is in kwargs
. A function executed on a thread process_excel_file
looks something like this:
def process_excel_file(ufile):
with temp_input_file(ufile) as temp_file:
in_book = xlrd.open_workbook(temp_file)
... # do stuff with in_book
So what happens when I use the context manager function temp_input_file
I get ValueError
because the stream is closed. If instead I refactor the code to create a temp file in the view and pass the file name temp instead of a file-like object, it works because the stream is closed somewhere after passing it to job.run_job
. Any thoughts on why the stream will be closed? This is more alarming because some files do not close the stream.
source to share
No one has answered this question yet
Check out similar questions: