Make a single numpy array column a different datatype
Given a numpy array my_arr
filled with strings, how do I set the datatype of one of the columns to be placed? I need this as a numpy array so that I can use it later with my existing code. See an example of a failed attempt below:
import numpy as np
dat = [['User1', 'Male', '2.2'], ['User2', 'Female', '3.777'], ['User3', 'Unknown', '0.0']]
my_arr = np.array(dat)
print my_arr
# [['User1' 'Male' '2.2'], ['User2' 'Female' '3.777'], ['User3' 'Unknown' '0.0']]
my_arr[:,2] = my_arr[:,2].astype(np.float)
print my_arr
# [['User1' 'Male' '2.2'], ['User2' 'Female' '3.777'], ['User3' 'Unknown' '0.0']]
source to share
There may be smarter ways to do this, but the following gives you the correct result, which I think; you can use structured arrays :
import numpy as np
dat = [['User1', 'Male', '2.2'], ['User2', 'Female', '3.777'], ['User3', 'Unknown', '0.0']]
# create data types: two strings of length 10 and float
dt = np.dtype('a10, a10, float')
# convert the inner lists to tuples so that a structured array can be used
for ind, l in enumerate(dat):
dat[ind] = tuple(l)
# convert dat to an array
my_arr = np.array(dat, dt)
Output:
array([('User1', 'Male', 2.2), ('User2', 'Female', 3.777),
('User3', 'Unknown', 0.0)],
dtype=[('f0', 'S10'), ('f1', 'S10'), ('f2', '<f8')])
You can also give names to columns by running:
dt = {'names': ['user', 'gender', 'number'], 'formats':['a10', 'a10', 'float']}
my_arr = np.array(dat, dt) # dat is the list with tuples, see above
Now the output is:
array([('User1', 'Male', 2.2), ('User2', 'Female', 3.777),
('User3', 'Unknown', 0.0)],
dtype=[('user', 'S10'), ('gender', 'S10'), ('number', '<f8')])
And then you can access one column like this
my_arr['number']
array([ 2.2 , 3.777, 0. ])
my_arr['user']
array(['User1', 'User2', 'User3'], dtype='|S10')
I would recommend using a dataframe from Python pandas where you can easily deal with different data types and complex data structures.
In your example:
import pandas as pd
pd.DataFrame(dat, columns=['user', 'gender', 'some number'])
will just give you:
user gender some number
0 User1 Male 2.2
1 User2 Female 3.777
2 User3 Unknown 0.0
source to share
You can convert your 2d array to a mixed structured array dtype
:
In [137]: my_arr
Out[137]:
array([['User1', 'Male', '2.2'],
['User2', 'Female', '3.777'],
['User3', 'Unknown', '0.0']],
dtype='<U7')
In [138]: dt=np.dtype('U7,U7,f') # complex dtype
In [139]: np.array([tuple(row) for row in my_arr], dtype=dt)
Out[139]:
array([('User1', 'Male', 2.200000047683716),
('User2', 'Female', 3.7769999504089355), ('User3', 'Unknown', 0.0)],
dtype=[('f0', '<U7'), ('f1', '<U7'), ('f2', '<f4')])
In [140]: _.shape
Out[140]: (3,)
It is now an array 1d
with three fields. Instead of accessing columns by number, you access fields by name, arr['f0']
etc.
I used [tuple(row) for row in my_arr]
because the input for structured arrays should be a list of tuples. I could use your list dat
, [tuple(row) for row in dat]
.
source to share