Clever Geek Handbook

Find indices of a DataFrame list with NaN values ​​- Pandas

I have a list of data frames in which some data frames have values NaN

. So far, I was able to identify values NaN

for one dataframe using this link.

How to find the index of a list in which a Data Frame has values NaN

.

List example dffs

,

[                   
 var1       var1  
14.171250  13.593813
13.578317  13.595329
10.301850  13.580139
9.930217   NaN
6.192517   13.561943
NaN        13.565149
6.197983   13.572509,  

  var1       var2    
2.456183  5.907528
5.052017  5.955731
5.960000  5.972480
8.039317  5.984608
7.559217  5.985348
6.933633  5.979438,

 var1       var1  
14.171250  23.593813
23.578317  23.595329
56.301850  23.580139
90.930217   22.365676
89.192517   33.561943
86.23654   53.565149
NaN        13.572509,  
...]

I need to get the results in a list indexes

0

and 2

that are relevant NaN

.

So far I have tried this,

df_with_nan = []
for df in dffs:
    df_with_nan.append(df.columns[df.isnull().any()])

In the above loop, for

I get the column names, var1

and var2

. However, I need the indices of these dataframes as I go through it. Any help or suggestion would be great.

+3


source to share


2 answers


You are almost there ... just use enumerate

for a loop with indices and df.isnull().values.any()

(faster than df.isnull().any().max()

) to check:

df_with_nan = []
for i, df in enumerate(dffs):
    if df.isnull().values.any():
        df_with_nan.append(i)


Of course the comp list is shorter, but go for whatever you prefer.

+1


source


You can use a conditional list comprehension to enumerate all data files in your list and return the indexed index value of those that contain any null values.



df_with_nan = [n for n, df in enumerate(dffs) if sum(df.isnull().any())]
+2


source






More articles:


All Articles