Re-shape pandas stack / data frame
df = pd.DataFrame({'BORDER':['GERMANY','FRANCE','ITALY','USA','CANADA','MEXICO','INDIA','CHINA','JAPAN' ], 'ASID':[21, 32, 99, 77,66,55,44,88,111], 'HOUR1':[2 ,2 ,2 ,4 ,4 ,4 ,6 ,6, 6],'HOUR2':[3 ,3 ,3, 5 ,5 ,5, 7, 7, 7], 'HOUR3':[8 ,8 ,8, 12 ,12 ,12, 99, 99, 99], 'PRICE1':[2 ,2 ,2 ,4 ,4 ,4 ,6 ,6, 6], 'PRICE2':[2 ,2 ,2 ,4 ,4 ,4 ,6 ,6, 6],'PRICE3':[2 ,2 ,2 ,4 ,4 ,4 ,6 ,6, 6] })
df = df[['ASID', 'BORDER', 'HOUR1', 'PRICE1', 'HOUR2', 'PRICE2', 'HOUR3', 'PRICE3']]
I have been trying to reformat this data file for the last day. Workout with stack / spread / melt and shift columns into signs, etc., but failed to achieve my goal.
The desired output has the following columns:
ASID, BORDER, HOUR, PRICE
I want to combine everything ['HOUR1', 'HOUR2', HOUR3']
into one column = HOUR
.
Likewise, I want to combine everything ['PRICE1', 'PRICE2', 'PRICE3']
in one column = PRICE
so that the value in that field is aligned with the corresponding value in the column HOUR
. There is a relationship between HOUR1
and PRICE1
, HOUR2
and PRICE2
, HOUR3
and PRICE3
.
I appreciate any guidance you can provide.
source to share
Initial data (correction of the "PRICE1" notice in the second line).
df = pd.DataFrame({'BORDER':['GERMANY','FRANCE','ITALY','USA','CANADA','MEXICO','INDIA','CHINA','JAPAN' ], 'ASID':[21, 32, 99, 77,66,55,44,88,111], 'HOUR1':[2 ,2 ,2 ,4 ,4 ,4 ,6 ,6, 6],'HOUR2':[3 ,3 ,3, 5 ,5 ,5, 7, 7, 7], 'HOUR3':[8 ,8 ,8, 12 ,12 ,12, 99, 99, 99], 'PRICE1':[2 ,2 ,2 ,4 ,4 ,4 ,6 ,6, 6], 'PRICE2':[2 ,2 ,2 ,4 ,4 ,4 ,6 ,6, 6],'PRICE3':[2 ,2 ,2 ,4 ,4 ,4 ,6 ,6, 6] })
df = df[['ASID', 'BORDER', 'HOUR1', 'PRICE1', 'HOUR2', 'PRICE2', 'HOUR3', 'PRICE3']]
First, set the index to ASID
and BORDER
.
df.set_index(['ASID', 'BORDER'], inplace=True)
Then create two DataFrames for prices and hours, stacking the results. Throw away hours and price levels from these summarized DataFrames.
prices = df[['PRICE1','PRICE2', 'PRICE3']].stack()
prices.index = prices.index.droplevel(2)
hours = df[['HOUR1', 'HOUR2', 'HOUR3']].stack()
hours.index = hours.index.droplevel(2)
Finally, merge these two DataFrames and rename the columns.
df_new = pd.concat([hours, prices], axis=1)
df_new.columns = ['HOUR', 'PRICE']
>>> df_new
HOUR PRICE
ASID BORDER
21 GERMANY 2 2
GERMANY 3 2
GERMANY 8 2
32 FRANCE 2 2
FRANCE 3 2
FRANCE 8 2
99 ITALY 2 2
ITALY 3 2
ITALY 8 2
77 USA 4 4
USA 5 4
USA 12 4
66 CANADA 4 4
CANADA 5 4
CANADA 12 4
55 MEXICO 4 4
MEXICO 5 4
MEXICO 12 4
44 INDIA 6 6
INDIA 7 6
INDIA 99 6
88 CHINA 6 6
CHINA 7 6
CHINA 99 6
111 JAPAN 6 6
JAPAN 7 6
JAPAN 99 6
source to share