Django StreamingHttpResponse format error

I have a simple Django view to download a file from Amazon s3. The test with saving the file locally was ok:

def some_view(request):

    res = s3.get_object(...)

    try:
        s3_file_content = res['Body'].read()
        with open("/Users/yanik/ololo.jpg", 'wb') as f:
            f.write(s3_file_content)
            # file saved and I can view it
    except:
        pass

      

When you switch to StreamingHttpResponse

I got the wrong file format (can't open) and even the wrong size (if the original has an image of 317kb, the output tree is about 620kb)

def some_view(request):

    res = s3.get_object(...)

    response = StreamingHttpResponse(res['Body'].read(), content_type=res['ContentType'])
    response['Content-Disposition'] = 'attachment;filename=' + 'ololo.jpg'
    response['ContentLength'] = res['ContentLength']
    return response

      

Tried many different settings but nothing has worked for me so far. The output file is broken.

UPDATE

I managed to get more information about the opening. If I change the method of writing files in the first example from mode 'wb'

to 'w'

, I have the same output as from StreamingHttpResponse

(the first view will generate the same broken file). So it looks like I have to tell the http header that my output is in the formatbinary

UPDATE (getting to the bottom of the problem)

Now I understand the problem. But there is still no solution . res['Body'].read()

returns bytes

type and StreamingHttpResponse

iterates through those bytes, which returns byte codes. So, my pretty incoming bytes are '...\x05cgr\xb8=:\xd0\xc3\x97U\xf4\xf3\xdc\xf0*\xd4@\xff\xd9'

forcibly converted to an array as: [ ... , 195, 151, 85, 244, 243, 220, 240, 42, 212, 64, 255, 217]

and then loaded as concatenated strings. Screenshot: http://take.ms/JQztk As you can see, the list items are at the end.

StreamingHttpResponse.make_bytes
"""Turn a value into a bytestring encoded in the output charset."""

      

+1


source to share


2 answers


Still not sure what's going on. But FileWrapper

from fooobar.com/questions/235154 / ... works great for boto3's answer type StreamingBody

.



Welcome if anyone has the courage to explain this fuzzy behavior.

0


source


https://docs.djangoproject.com/en/1.9/ref/request-response/#streaminghttpresponse-objects

StreamingHttpResponse needs an iterator. I think if your file is binary (image) then StreamingHttpResponse is not the best solution, or you should create chunks of this file.

Bytearray is an iterator, but perhaps you want to go line by line not by byte / character.

I'm not sure if your file is string based text data, but if it is, you can create a generator to iterate over the file as an object:



def line_generator(file_like_obj):
    for line in file_like_obj:
        yield line

      

and feed this generator to StreamingHttpResponse:

some_view(request):
    res = s3.get_object(...)
    response = StreamingHttpResponse(line_generator(res['Body']), ...)
    return response

      

0


source







All Articles