Clever Geek Handbook

Merging and filling two data frames in Pandas at different time intervals

I have two dataframes that I would like to combine in Pandas. Both of them have a datetime column that I am merging with, however each has rows every minute and 5 minutes (depending on the year), while the other block of data has rows every 15 minutes. If I do an outer join I can concatenate them, but only rows every 15 minutes will have data from both data frames. I would like to copy 15 minute data and fill each of 1 or 5 minute rows with this data. Thus, data from 12:00 AM will fill all lines up to 12:14 AM. Then, 12:15 pm will be copied and filled in until 12:29 am, etc. Does this make sense? How can i do this?This question seems to be similar, but I'm not sure how to implement it for my exact scenario, especially considering that my one framework changes from 5 minutes to 1 minute intervals over the years.

+3


source to share


2 answers


You want to resample two data so that they have the same spacing and filling using the 'ffill' method



df1 = df1.resample('m').fillna(method = 'ffill')
df2 = df2.resample('m').fillna(method = 'ffill')
+1


source


Answering an old question, hoping to help others.

I had a similar problem, but besides the two timestamped dataframes, I had an additional column that determined whether 15-15 minutes data could match or not match my 1 ~ 1 min data. It looked like the key used for the ON option from the merge, but since I had this extra data, I couldn't use pd.merge directly.

I found this function solved directly, including some additional information using a parameter.



https://pandas.pydata.org/pandas-docs/stable/generated/pandas.merge_asof.html

The example in the docs is very simple.

+1


source






More articles:


All Articles