Why does the io.BytesIO stream close when passed as an argument?

I have a Django app where the user is uploading a file. The view grabs the file, passes it to the stream function, and processes it. The problem is that the io.BytesIO stream is closed before it can be read, and I cannot figure out why it closes.

I am processing Excel files using the Xlrd library. For some Excel downloads, the stream remains open, but some are close. I don't understand what is going on under the hood to understand why. What I'm asking is debug advice. I cannot provide my complete code, but I have outlined what the code looks like below.

My Django view grabs the file from the POST request:

ufile = request.FILES['upload-file'].file

      

ufile

is an instance io.BytesIO

. I have a function that copies a stream to a named temporary file

@contextmanager
def temp_input_file(file_):
  temp = tempfile.NamedTemporaryFile(delete=False)
  shutil.copyfileobj(file_, temp)
  temp.close()
  yield temp.name
  os.unlink(temp.name)

      

This decoration function of the contextmanager is used a bit later. The next step is to create a stream to process the uploaded file:

  job.run_job(
    method=process_excel_file,
    uploaded_file=ufile
  )

      

What I am doing is pass the function process_excel_file

and the uploaded "file", but actually the stream io.BytesIO

to job.run_job that creates the stream

def run_job(self, method, **kwargs):
  thread = threading.Thread()
  thread.run = lambda : method(
    job_id=self.job_db_record.id,
    **kwargs
  )
  thread.start()

      

So, it process_excel_file

is passed to a variable method

and the ufile is in kwargs

. A function executed on a thread process_excel_file

looks something like this:

def process_excel_file(ufile):
  with temp_input_file(ufile) as temp_file:
    in_book = xlrd.open_workbook(temp_file)
    ... # do stuff with in_book

      

So what happens when I use the context manager function temp_input_file

I get ValueError

because the stream is closed. If instead I refactor the code to create a temp file in the view and pass the file name temp instead of a file-like object, it works because the stream is closed somewhere after passing it to job.run_job

. Any thoughts on why the stream will be closed? This is more alarming because some files do not close the stream.

+3


source to share





All Articles