Python - Apply SciPy Beta Distribution to all rows of Pandas DataFrame
In SciPy, you can implement a beta distribution like this:
x=640495496
alpha=1.5017096
beta=628.110247
A=0
B=148000000000
p = scipy.stats.beta.cdf(x, alpha, beta, loc=A, scale=B-A)
Now, suppose I have a Pandas framework with columns x, alpha, beta, A, B. How do I apply the beta distribution to each row by adding the result to a new column?
+3
source to share
2 answers
Given that I suspect pandas apply is just a loop through all the lines, and there is quite a bit of overhead in scipy.stats distributions with each call, I would use the vectorized version:
>>> from scipy import stats
>>> df['p'] = stats.beta.cdf(df['x'], df['alpha'], df['beta'], loc=df['A'], scale=df['B']-df['A'])
>>> df
A B alpha beta x p
0 0 148000000000 1.501710 628.110247 640495496 0.858060
1 0 148000000000 1.501704 620.110000 640495440 0.853758
+1
source to share
Needed apply
with a function scipy.stats.beta.cdf
and axis=1
:
df['p'] = df.apply(lambda x: scipy.stats.beta.cdf(x['x'],
x['alpha'],
x['beta'],
loc=x['A'],
scale=x['B']-x['A']), axis=1)
Example:
import scipy.stats
df = pd.DataFrame({'x':[640495496, 640495440],
'alpha':[1.5017096,1.5017045],
'beta':[628.110247, 620.110],
'A':[0,0],
'B':[148000000000,148000000000]})
print (df)
A B alpha beta x
0 0 148000000000 1.501710 628.110247 640495496
1 0 148000000000 1.501704 620.110000 640495440
df['p'] = df.apply(lambda x: scipy.stats.beta.cdf(x['x'],
x['alpha'],
x['beta'],
loc=x['A'],
scale=x['B']-x['A']), axis=1)
print (df)
A B alpha beta x p
0 0 148000000000 1.501710 628.110247 640495496 0.858060
1 0 148000000000 1.501704 620.110000 640495440 0.853758
+2
source to share