First value in Pandas DatetimeIndex not recognized
I have a simple series:
>>> sub_dim_metrics
date
2017-04-04 00:00:00+00:00 32.38
2017-04-03 00:00:00+00:00 246.28
2017-04-02 00:00:00+00:00 146.25
2017-04-01 00:00:00+00:00 201.98
2017-03-31 00:00:00+00:00 274.74
2017-03-30 00:00:00+00:00 257.82
2017-03-29 00:00:00+00:00 279.38
2017-03-28 00:00:00+00:00 203.53
2017-03-27 00:00:00+00:00 250.65
2017-03-26 00:00:00+00:00 180.59
2017-03-25 00:00:00+00:00 196.61
2017-03-24 00:00:00+00:00 281.04
2017-03-23 00:00:00+00:00 276.44
2017-03-22 00:00:00+00:00 227.55
2017-03-21 00:00:00+00:00 267.59
Name: area, dtype: float64
>>> sub_dim_metrics.index
DatetimeIndex(['2017-04-04', '2017-04-03', '2017-04-02', '2017-04-01',
'2017-03-31', '2017-03-30', '2017-03-29', '2017-03-28',
'2017-03-27', '2017-03-26', '2017-03-25', '2017-03-24',
'2017-03-23', '2017-03-22', '2017-03-21'],
dtype='datetime64[ns, UTC]', name=u'date', freq=None)
Later in my code, I get the scope for specific days using the following format: sub_dim_metrics['2017-04-02']
eg.
Before I get the scope for a specific day, I first check that the requested date is in a Series, for example: if '2017-04-02' in sub_dim_metrics.index
My problem is that the first value in the index doesn't return true, while the rest do:
>>> '2017-04-02' in sub_dim_metrics.index
True
>>> '2017-04-04' in sub_dim_metrics.index
False
Why is this and what is the best way to check the date in my series before getting its corresponding value?
source to share
IIUC:
You get False
when you expect True
:
You check if the string is in the datetime index. Apparently pandas
free of verification and trying to do it for you. It gets wrong though, doesn't it.
plan 1
Correct!
pd.to_datetime('2017-04-04') in sub_dim_metrics.index
True
plan 2
I think unsorted-ness is discarding it. sort_values
...
'2017-04-04' in sub_dim_metrics.index.sort_values()
True
customization
from io import StringIO
import pandas as pd
txt = """2017-04-04 00:00:00+00:00 32.38
2017-04-03 00:00:00+00:00 246.28
2017-04-02 00:00:00+00:00 146.25
2017-04-01 00:00:00+00:00 201.98
2017-03-31 00:00:00+00:00 274.74
2017-03-30 00:00:00+00:00 257.82
2017-03-29 00:00:00+00:00 279.38
2017-03-28 00:00:00+00:00 203.53
2017-03-27 00:00:00+00:00 250.65
2017-03-26 00:00:00+00:00 180.59
2017-03-25 00:00:00+00:00 196.61
2017-03-24 00:00:00+00:00 281.04
2017-03-23 00:00:00+00:00 276.44
2017-03-22 00:00:00+00:00 227.55
2017-03-21 00:00:00+00:00 267.59"""
sub_dim_metrics = pd.read_csv(StringIO(txt),
sep='\s{2,}', engine='python',
index_col=0, parse_dates=[0],
header=None, names=['date', 'area'],
squeeze=True)
source to share