196
$\begingroup$

I have been using pandas for quite some time. But, I don't understand what's the difference between isna() and isnull(). And, more importantly, which one to use when identifying missing values in a dataframe.

What is the basic underlying difference of how a value is detected as either na or null?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ The two functions are same. Both give the missing values. $\endgroup$
    – Ankit Seth
    Commented Sep 6, 2018 at 11:37
  • $\begingroup$ Both perform the same task of detecting null values. $\endgroup$
    – R Gupta
    Commented Oct 1, 2022 at 17:21

2 Answers 2

Reset to default
240
$\begingroup$

Pandas isna() vs isnull().

I'm assuming you are referring to pandas.DataFrame.isna() vs pandas.DataFrame.isnull(). Not to confuse with pandas.isnull(), which in contrast to the two above isn't a method of the DataFrame class.

These two DataFrame methods do exactly the same thing! Even their docs are identical. You can even confirm this in pandas' code.

But why have two methods with different names do the same thing?

This is because pandas' DataFrames are based on R's DataFrames. In R na and null are two separate things. Read this post for more information.

However, in python, pandas is built on top of numpy, which has neither na nor null values. Instead numpy has NaN values (which stands for "Not a Number"). Consequently, pandas also uses NaN values.

In short

  • To detect NaN values numpy uses np.isnan().

  • To detect NaN values pandas uses either .isna() or .isnull().
    The NaN values are inherited from the fact that pandas is built on top of numpy, while the two functions' names originate from R's DataFrames, whose structure and functionality pandas tried to mimic.

Improve this answer
$\endgroup$
3
  • 9
    $\begingroup$ This explains everything and yes i wanted to infer 'pandas.DataFrame.isna()' vs 'pandas.DataFrame.isnull()' . Thanks for such a detailed explanation. $\endgroup$
    – Vaibhav Thakur
    Commented Sep 7, 2018 at 6:45
  • 3
    $\begingroup$ Do note that in Pandas 1.0.0 a new experimental pd.NA value has been introduced and it might behave differently in certain operations from np.nan. $\endgroup$
    – Bruno Lubascher
    Commented Apr 21, 2020 at 14:27
  • $\begingroup$ Can you also confirm that df[df['x'].notnull()] is the same as df[~df['x'].isna()] & df[~df['x'].isnull()]? Only recently seen notnull and want to be sure its safe $\endgroup$
    – Olivia
    Commented Oct 25, 2022 at 10:12
0
$\begingroup$

isnull is an alias for isna, so they are the same. You can check the source code to confirm as much.

Similarly, notnull is an alias for notna, which is defined as ~df.isna() (source code), so the following are all equivalent:

df[df['x'].notnull()]

df[df['x'].notna()]

df[~df['x'].isna()]

df[~df['x'].isnull()]
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.