Save pandas dataframe as 32bit float
Use numpy.float32
:
In [320]:
import numpy as np
import pandas as pd
df = pd.DataFrame({'a':np.random.randn(10)})
df.info()
<class 'pandas.core.frame.DataFrame'>
Int64Index: 10 entries, 0 to 9
Data columns (total 1 columns):
a 10 non-null float64
dtypes: float64(1)
memory usage: 160.0 bytes
In [323]:
df['a'].astype(np.float32)
Out[323]:
0 0.966618
1 -0.331942
2 0.906349
3 -0.089582
4 -0.722004
5 0.668103
6 0.230314
7 -1.707631
8 1.806862
9 1.783765
Name: a, dtype: float32
You can see the dtype is now float32
source to share
Just continuing the accepted answer. Note: If memory is limited, or if you need more space, you can choose df['a'].astype(np.float32)
as an answer gives or equally substitute np.float16
or np.float64
for the numbers, np.int16
, np.int32
, np.int64
for integers, in many applications, you can cut up to int16
/ float16
and reduce the size of your data, if the precision your application will be good.
source to share