How to filter in NaN (pandas)?
I have a pandas dataframe (df), and I want to do something like:
newdf = df[(df.var1 == 'a') & (df.var2 == NaN)]I've tried replacing NaN with np.NaN, or 'NaN' or 'nan' etc, but nothing evaluates to True. There's no pd.NaN.
I can use df.fillna(np.nan) before evaluating the above expression but that feels hackish and I wonder if it will interfere with other pandas operations that rely on being able to identify pandas-format NaN's later.
How pandas filter nan? Any advice is appreciated. Thank you.
When NaN is being compared to itself it returns false. You can use pd.isnull(df.var2) instead of NaN.
Note: To filter out the rows of pandas dataframe that has missing values in Last_Namecolumn, we will first find the index of the column with non null values with pandas notnull() function. It will return a boolean series, where True for not null and False for null values or missing