Pandas dataframe tuples?
I have a pandas framework that I create from a list (which is generated from the spark of rdd) by calling:
newRdd = rdd.map(lambda row: Row(row.__fields__ + ["tag"])(row + (tagScripts(row), ))).collect()
, and then df = pd.DataFrame(newRdd)
My data looks like dataframe of tuples as shown below:
0 (2017-06-21, Sun, ATL, 10)
1 (2017-06-21, Sun, ATL, 11)
2 (2017-06-21, Sun, ATL, 11)
but I need it to look like a standard table with column headings as such:
date dayOfWeek airport val1
2017-06-11 Sun ATL 11
I am honest out of ideas about this and need some help. I tried many different things and nothing worked. Any help would be greatly appreciated. Thank you for your time.
You can do it like this:
df = pd.DataFrame([*df.A],columns = ['date','dayOfWeek','airport','val1','val2','val3','val4','val5','val6'])
I assumed the name of the column is in an already existing data file A
.
you can check here for unpacking tuples.
Hope this was helpful. there are questions about this, please let me know.
source to share