Slicing in PyTables

What is the fastest way to slice through arrays stored in h5 using PyTables?

The scenario is as follows:

  • The data has already been saved (no need to optimize here):

    filters = tables.Filters(complib='blosc', complevel=5)
    h5file = tables.open_file(hd5_filename, mode='w',
                              title='My Data',
                              filters=filters)
    group = h5file.create_group(h5file.root, 'Data', 'Data')
    
    X_atom = tables.Float32Atom(shape=[50,50,50])
    X = h5file.create_carray(group, 'X', atom=X_atom, title='XData',
                         shape=(1000,), filters=filters)
    
          

  • Data is opened:

    h5file = tables.openFile(hd5_filename, mode="r")
    node = h5file.getNode('/', data_node)
    X = getattr(node, X_str)
    
          

  • This is where I need optimization, I need to do many of the following kinds of array slice, which cannot be sorted, for many indices and different min / max locations:

    for index, min_x, min_y, min_z, max_x, max_y, max_z in my_very_long_list:    
        current_item = X[index][min_x:max_x,min_y:max_y,min_z:max_z]
        do_something(current_item)
    
          

Question: Is this the fastest way to complete the task?

+3


source to share





All Articles