IOError: Cannot read data (Cannot open directory) - Missing gzip compression filter
I've never worked with HDF5 files before, and I got some sample files to start with. I checked all the basics with help h5py
by looking at the different groups in these files, their names, keys, values, etc. Everything works fine until I take a look at the data being saved in groups. I get them .shape
and .dtype
, but when I try to access the random value by indexing (for example grp["dset"][0]
), I get the following error:
IOError Traceback (most recent call last)
<ipython-input-45-509cebb66565> in <module>()
1 print geno["matrix"].shape
2 print geno["matrix"].dtype
----> 3 geno["matrix"][0]
/home/sarah/anaconda/lib/python2.7/site-packages/h5py/_hl/dataset.pyc in __getitem__(self, args)
443 mspace = h5s.create_simple(mshape)
444 fspace = selection._id
--> 445 self.id.read(mspace, fspace, arr, mtype)
446
447 # Patch up the output for NumPy
/home/sarah/anaconda/lib/python2.7/site-packages/h5py/h5d.so in h5py.h5d.DatasetID.read (h5py/h5d.c:2782)()
/home/sarah/anaconda/lib/python2.7/site-packages/h5py/_proxy.so in h5py._proxy.dset_rw (h5py/_proxy.c:1709)()
/home/sarah/anaconda/lib/python2.7/site-packages/h5py/_proxy.so in h5py._proxy.H5PY_H5Dread (h5py/_proxy.c:1379)()
IOError: Can't read data (Can't open directory)
I posted this issue on the h5py google group where it was suggested that there might be a filter in the dataset, t. But the HDF5 file was created using gzip compression only, which I understood should be a portable standard.
Does anyone know what I'm missing here? I can't even find a description of this error or similar issues anywhere, and the file, including the problematic dataset, can be easily opened with HDFView software.
Edit
Apparently this error occurs because the gzip compression filter is not available on my system for some reason. If I try to create an example file with gzip compression, this happens:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-33-dd7b9e3b6314> in <module>()
1 grp = f.create_group("subgroup")
----> 2 grp_dset = grp.create_dataset("dataset", (50,), dtype="uint8", chunks=True, compression="gzip")
/home/sarah/anaconda/lib/python2.7/site-packages/h5py/_hl/group.pyc in create_dataset(self, name, shape, dtype, data, **kwds)
92 """
93
---> 94 dsid = dataset.make_new_dset(self, shape, dtype, data, **kwds)
95 dset = dataset.Dataset(dsid)
96 if name is not None:
/home/sarah/anaconda/lib/python2.7/site-packages/h5py/_hl/dataset.pyc in make_new_dset(parent, shape, dtype, data, chunks, compression, shuffle, fletcher32, maxshape, compression_opts, fillvalue, scaleoffset, track_times)
97
98 dcpl = filters.generate_dcpl(shape, dtype, chunks, compression, compression_opts,
---> 99 shuffle, fletcher32, maxshape, scaleoffset)
100
101 if fillvalue is not None:
/home/sarah/anaconda/lib/python2.7/site-packages/h5py/_hl/filters.pyc in generate_dcpl(shape, dtype, chunks, compression, compression_opts, shuffle, fletcher32, maxshape, scaleoffset)
101
102 if compression not in encode:
--> 103 raise ValueError('Compression filter "%s" is unavailable' % compression)
104
105 if compression == 'gzip':
ValueError: Compression filter "gzip" is unavailable
Does anyone have any experience? Installing HDF5 library as well as h5py package didn't seem to go wrong ...
source to share
I had a similar problem,
$ python3 -c 'import h5py; f=h5py.File("file.h5"); d=f["FVC"][:,:]'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/home/ilan/minonda/conda-bld/work/h5py/_objects.c:2696)
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/home/ilan/minonda/conda-bld/work/h5py/_objects.c:2654)
File "/home/pinaultf/system/anaconda2/envs/deveg-dev/lib/python3.5/site-packages/h5py/_hl/dataset.py", line 482, in __getitem__
self.id.read(mspace, fspace, arr, mtype, dxpl=self._dxpl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/home/ilan/minonda/conda-bld/work/h5py/_objects.c:2696)
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/home/ilan/minonda/conda-bld/work/h5py/_objects.c:2654)
File "h5py/h5d.pyx", line 181, in h5py.h5d.DatasetID.read (/home/ilan/minonda/conda-bld/work/h5py/h5d.c:3240)
File "h5py/_proxy.pyx", line 130, in h5py._proxy.dset_rw (/home/ilan/minonda/conda-bld/work/h5py/_proxy.c:1869)
File "h5py/_proxy.pyx", line 84, in h5py._proxy.H5PY_H5Dread (/home/ilan/minonda/conda-bld/work/h5py/_proxy.c:1517)
OSError: Can't read data (Can't open directory)
I had this problem in one virtual environment and not another, even if apparently the h5py version was the same (2.6.0).
I solved this problem:
$ pip uninstall h5py $ pip install h5py
source to share