Function vectorization in pandas

You don't need anything interesting, just a few mods for your function.

First, don't use a library math

. If you are doing real math or science, you will probably be better off with numpy.

Secondly, we will use the dataframe method apply

. What it does apply

is that it executes a function and runs each row (axis = 1) or column (axis = 0) through it and creates a new pandas object with all return values. So we need to set up the haversine

totake line in the dataframe and unpack the values. It will be:

def haversine(row):
    """
    Calculate the great circle distance between two points 
    on the earth (specified in decimal degrees)
    """
    import numpy as np

    # convert all of the row to radians
    row = np.radians(row)

    # unpack the values for convenience
    lat1 = row['lat1']
    lat2 = row['lat2']
    lon1 = row['lon1']
    lon2 = row['lon2']

    # haversine formula 
    dlon = lon2 - lon1 
    dlat = lat2 - lat1 
    a = np.sin(dlat/2)**2 + np.cos(lat1) * np.cos(lat2) * np.sin(dlon/2)**2
    c = 2 * np.arcsin(np.sqrt(a)) 

    # 6367 km is the radius of the Earth
    km = 6367 * c
    return km

      

        
        
        
      

    

Ok, now we need to get the form data in the form. In your question, it is all string and this is bad for math. So using your variable d

I said:

df = pandas.DataFrame(d).set_index('Provider ID').astype(float)

      

        
        
        
      

    

Thus created a dataframe of rows, set the provider as the index and then convert all columns to floats as we are doing the math.

Now we need to create strings with two sets of coordinates. For this we will use the method shift

and attach the result to the original file frame. Doing it straight away looks like this:

df = df.join(df.shift(), lsuffix='1', rsuffix='2')
print(df.head())

                  lat1       lon1       lat2       lon2
Provider ID                                            
10001        31.215379 -85.361466        NaN        NaN
10005        34.221335 -86.159375  31.215379 -85.361466
10006        34.795040 -87.685075  34.221335 -86.159375
10007        31.292160 -86.255399  34.795040 -87.685075
10008        31.693116 -86.265495  31.292160 -86.255399

      

        
        
        
      

    

rsuffix

and lsuffix

is what adds "1" and "2" to the column names during the join operation.

Columns "2" are from df.shift()

, and you will notice that they are equal to columns "1" of the previous row. You will also see that the first row of columns "2" is NaN

because nothing precedes the first row.

So now we can have apply

the Haversine function:

distance = df.apply(haversine, axis=1)
print(distance)
Provider ID
10001                 NaN
10005          342.261590
10006          153.567591
10007          411.393751
10008           44.566642
10011          214.661170
10012          125.775583
10016          163.973219
10018           27.659157
10019          160.901128
dtype: float64