Clever Geek Handbook

Speeding up column conversions

Let's say I have a large data size and I want to apply one operation to each item in a column.

Is there a faster way to do this than the following:

get_weekday = lambda x: time.strptime(str(x), '%d%m%Y').tm_wday
df['date'] = df['date'].apply(get_weekday)

?

+3


source to share


1 answer


In the current main /0.15.0

df['date'].dt.weekday

In previous versions



pd.DatetimeIndex(df['date']).weekday

Here's a sync example

In [16]: s = Series(date_range('20130101',freq='s',periods=100000))

In [17]: %timeit s.dt.weekday
10 loops, best of 3: 50.8 ms per loop

In [18]: s2 = s.apply(str)

In [19]: %timeit s.apply(lambda x: time.strptime(str(x), "%Y-%m-%d %H:%M:%S").tm_wday)
1 loops, best of 3: 2.65 s per loop
+5


source






More articles:


All Articles