Compare ctypes arrays without extra memory
I have two large arrays of ctypes that I would like to compare, without additional memory. Direct comparison doesn't work:
>>> a = ctypes.create_string_buffer(b'1'*0x100000)
>>> b = ctypes.create_string_buffer(b'1'*0x100000)
>>> a == b
False
Using the attribute value
or raw
creates a copy of the array in memory.
Using memoryview
both buffers for transfer slows things down a lot.
For Windows, a possible solution is to use msvcrt.memcmp
directly, but is there a more pythonic way or a cross-platform way to do this?
source to share
Specific C libraries can be found on the platform using ctypes.util.find_library
. The functions provided by the library can be used at will.
Thus, arrays can be compared by doing the following:
libc_name = ctypes.util.find_library("c")
libc = ctypes.CDLL(libc_name)
libc.memcmp.argtypes = (ctypes.c_void_p, ctypes.c_void_p, ctypes.c_size_t)
len(a) == len(b) and libc.memcmp(a, b, len(a)) == 0
Be warned, these function calls are very unforgiving if called incorrectly. By setting the function argips, you make the function check its parameters before calling the library function.
A pure pythonic way to compare arrays without using large additional memory is as follows. It uses a generator to compare each element at a time, rather than copying all arrays elsewhere and then comparing them.
len(a) == len(b) and all(x == y for x, y in zip(a,b))
The disadvantage of this is that many objects will be created, each with a small amount of memory, which will fall on its own computational cost (instead of the processor, not memory).
source to share