"IndexError: Positional indexes are out of bounds" when they are clearly not
Here's an MWE of some code I'm using. I slowly strip away the original dataframe with slicing and some conditions until I only have the rows I need. Each block of five lines is actually a different object, so when I minify everything, if any one line in each block of five meets the criteria, I want to keep it - that's what ends the loop over keep.index. No matter what, when I am done, I can see that the end indexes I want exist, but I get an "IndexError: Positional indexes out of bounds" error. What's going on here?
import pandas as pd
import numpy as np
temp = np.random.rand(100,5)
df = pd.DataFrame(temp, columns=['First', 'Second', 'Third', 'Fourth', 'Fifth'])
df_cut = df.iloc[10:]
keep = df_cut.loc[(df_cut['First'] < 0.5) & (df_cut['Second'] <= 0.6)]
new_indices_to_use = []
for item in keep.index:
remainder = (item % 5)
add = np.arange(0-remainder,5-remainder,1)
inds_to_use = item + add
new_indices_to_use.append(inds_to_use)
new_indices_to_use = [ind for sublist in new_indices_to_use for ind in sublist]
final_indices_to_use = []
for item in new_indices_to_use:
if item not in final_indices_to_use:
final_indices_to_use.append(item)
final = df_cut.iloc[final_indices_to_use]
source to share
From the Pandas documentation on .iloc
(emphasis mine):
Pandas provides a set of methods for getting pure integer indexing. The semantics are closely related to python and numpy slicing. This is 0 indexing .
You are trying to use it by label, which means you need .loc
In your example:
>>>print df_cut.iloc[89]
...
Name: 99, dtype: float64
>>>print df_cut.loc[89]
...
Name: 89, dtype: float64
source to share