Error calculating axis in nanmax while nansum works in the same case
I am trying to get the maximum number of numpy.arrays as in the following code:
[np.nanmax(temp_data[i:i+window_size, 1:], axis=0) for i, t in enumerate(temp_data)]
Although, I get the following error:
{TypeError}ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
Anyway, if instead of nanmax
, I use nansum
like in the following code, everything works smoothly:
[np.nansum(temp_data[i:i+window_size, 1:], axis=0) for i, t in enumerate(temp_data)]
=> A 9x2 requested np.array
Also, for codes with nanmax
or nansum
both work:
[np.nansum(temp_data[i:i+window_size, 1:]) for i, t in enumerate(temp_data)]
[np.nanmax(temp_data[i:i+window_size, 1:]) for i, t in enumerate(temp_data)]
=> A 9x1 np.array, (but this is not what I want)
Any idea why specifying the axis in nanmax
fails while it works for nansum
?
Edit: example temp_data:
temp_data = np.array([[datetime.datetime(1980, 1, 1, 0, 0), np.nan, np.nan],
[datetime.datetime(1980, 1, 2, 0, 0), np.nan, np.nan],
[datetime.datetime(1980, 1, 3, 0, 0), np.nan, np.nan],
[datetime.datetime(1980, 1, 4, 0, 0), np.nan, np.nan],
[datetime.datetime(1980, 1, 7, 0, 0), np.nan, 1],
[datetime.datetime(1980, 1, 8, 0, 0), np.nan, 2],
[datetime.datetime(1980, 1, 9, 0, 0), 1, 3],
[datetime.datetime(1980, 1, 10, 0, 0), 5, 4],
[datetime.datetime(1980, 1, 11, 0, 0), 4, 1]])
source to share
You can force the conversion float
to data as numpy doesn't seem to know how to handle data when it's of type object
:
# \/
>>> [np.nanmax(temp_data[i:i+window_size, 1:].astype(float), axis=0) for i, t in enumerate(temp_data)]
[array([ nan, nan]),
array([ nan, nan]),
array([ nan, nan]),
array([ nan, 1.]),
array([ nan, 2.]),
array([ 1., 3.]),
array([ 5., 4.]),
array([ 5., 4.]),
array([ 4., 1.])]
source to share
This is either a bug or the result of undefined behavior in numpy. Your array temp_data
has a dtype object
. This means that the values in the array can be arbitrary Python objects. The error you see happens when you pass an array of objects nanmax
and specify an axis.
Here's a simple example using numpy version 1.12.1:
In [21]: a = np.array([[1.0, 2.0], [3.0, 4.0]], dtype=object)
In [22]: np.nanmax(a, axis=0)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-22-a020f98a2536> in <module>()
----> 1 np.nanmax(a, axis=0)
/Users/warren/miniconda3scipy/lib/python3.5/site-packages/numpy/lib/nanfunctions.py in nanmax(a, axis, out, keepdims)
343 # Fast, but not safe for subclasses of ndarray
344 res = np.fmax.reduce(a, axis=axis, out=out, **kwargs)
--> 345 if np.isnan(res).any():
346 warnings.warn("All-NaN slice encountered", RuntimeWarning, stacklevel=2)
347 else:
TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
The function does not raise an error if axis
not specified:
In [23]: np.nanmax(a)
Out[23]: 4.0
nansum()
handles the axis correctly:
In [24]: np.nansum(a, axis=0)
Out[24]: array([4.0, 6.0], dtype=object)
If the objects in the array are all converting to Python floating point values, you can use the method astype()
to convert the array to one with the data type numpy.float64
for which nanmax()
works as expected:
In [26]: a.astype(np.float64)
Out[26]:
array([[ 1., 2.],
[ 3., 4.]])
In [27]: np.nanmax(a.astype(np.float64), axis=0)
Out[27]: array([ 3., 4.])
source to share