12
$\begingroup$

I have two data frames. So how do I remove rows that have identical email addresses in df2 from df1?

>>df1
   First  Last  Email
0 Adam   Smith  email@email.com
1 John   Brown  email2@email.com
2 Joe    Max    email3@email.com
3 Will   Bill   email4@email.com


>>df2
  First  Last   Email
0 Adam   Smith  email@email.com
1 John   Brown  email2@email.com
$\endgroup$
0

2 Answers 2

14
$\begingroup$

You can try this:

cond = df1['Email'].isin(df2['Email'])
df1.drop(df1[cond].index, inplace = True)

>>df1
    First   Last    Email
2   Joe     Max     email3@email.com
3   Will    Bill    email4@email.com
$\endgroup$
1
  • $\begingroup$ This method does not work if you don't have a unique column (email in this case). In this case you can join two or more columns into one with df1[['First', 'Last']].agg('-'.join, axis=1) if they are strings. $\endgroup$
    – alvitawa
    Apr 3 at 12:47
6
$\begingroup$

Simpler to use isin() with dropna()

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.isin.html

df1[~df1.isin(df2)].dropna()
$\endgroup$
3
  • $\begingroup$ Quick question, what does the tilde stands for? $\endgroup$ Apr 23, 2021 at 18:23
  • $\begingroup$ It's a negation. Will turn all True to False and vice versa. $\endgroup$
    – Idodo
    Apr 26, 2021 at 8:50
  • $\begingroup$ This is wrong. It compares the values one at a time, a row can have mixed cases. Even when a row has all true, that doesn't mean that same row exists in the other dataframe, it means the values of this row exist in the columns of the other dataframe but in multiple rows. $\endgroup$
    – anishtain4
    May 13 at 17:08

Not the answer you're looking for? Browse other questions tagged or ask your own question.