How to get 2d numpy array from pandas dataframe? - irregular shape
I want to get a 2d-numpy array from a pandas dataframe column df
with a numpy vector on each row. But if I do
df.values.shape
I get: (3,)
instead of getting:(3,5)
(assuming each numpy vector in the dataframe has 5 dimensions and that there are 3 rows in the dataframe)
What is the correct method?
source to share
Ideally, avoid getting into this situation by looking for another way to define the DataFrame in the first place. However, if your DataFrame looks like this:
s = pd.Series([np.random.randint(20, size=(5,)) for i in range(3)])
df = pd.DataFrame(s, columns=['foo'])
# foo
# 0 [4, 14, 9, 16, 5]
# 1 [16, 16, 5, 4, 19]
# 2 [7, 10, 15, 13, 2]
then you can convert it to a DataFrame of the form (3,5) by calling pd.DataFrame
on the array list:
pd.DataFrame(df['foo'].tolist())
# 0 1 2 3 4
# 0 4 14 9 16 5
# 1 16 16 5 4 19
# 2 7 10 15 13 2
pd.DataFrame(df['foo'].tolist()).values.shape
# (3, 5)
source to share
I'm not sure what you want. But df.values.shape
it seems to give the correct result.
import pandas as pd
import numpy as np
from pandas import DataFrame
df3 = DataFrame(np.random.randn(3, 5), columns=['a', 'b', 'c', 'd', 'e'])
print df3
# a b c d e
#0 -0.221059 1.206064 -1.359214 0.674061 0.547711
#1 0.246188 0.628944 0.528552 0.179939 -0.019213
#2 0.080049 0.579549 1.790376 -1.301700 1.372702
df3.values.shape
#(3L, 5L)
df3["a"]
#0 -0.221059
#1 0.246188
#2 0.080049
df3[:1]
# a b c d e
#0 -0.221059 1.206064 -1.359214 0.674061 0.547711
source to share