Reading binary file of unknown size with mixed data types in python
I need to read binaries consisting of 19 float32
numbers followed by an unknown number uint32
. How can I read a file like this in python?
In Matlab, the equivalent looks like this:
fid = fopen('myFile.bin','r'); params = fread(fid,19,'float'); data = fread(fid,'uint32'); fclose(fid);
source to share
Use numpy.fromfile()
method and pass it a file with the appropriate number of elements read.
import numpy as np
with open('myFile.bin', 'rb') as f:
params = np.fromfile(f, dtype=np.float32, count=19)
data = np.fromfile(f, dtype=np.int32, count=-1) # I *assumed* here your ints are 32-bit
.tolist()
Let's open up paranthesis before closing fromfile()
(ex:) np.fromfile(...).tolist()
if you want to get standard Python lists instead of numpy
arrays.
source to share
For reading a binary file, I recommend using struct package
The solution can be written as follows:
import struct
f = open("myFile.bin", "rb")
floats_bytes = f.read(19 * 4)
# here I assume that we read exactly 19 floats without errors
# 19 floats in array
floats_array = struct.unpack("<19f", floats_bytes)
# convert to list beacause struct.unpack returns tuple
floats_array = list(floats_array)
# array of ints
ints_array = []
while True:
int_bytes = r.read(4)
# check if not eof
if not int_bytes:
break
int_value = struct.unpack("<I", int_bytes)[0]
ints_array.append(int_value)
f.close()
Note that I assumed that your numbers are in least significant byte order, so I used "<" in the format strings.
source to share