17
$\begingroup$

I have two data frames df1 and df2. For my analysis, I need to remove rows from df1 that have identical column values (Email) in df2?

>>df1
   First  Last  Email
0 Adam   Smith  [email protected]
1 John   Brown  [email protected]
2 Joe    Max    [email protected]
3 Will   Bill   [email protected]


>>df2
  First  Last   Email
0 Adam   Smith  [email protected]
1 John   Brown  [email protected]
Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
17
$\begingroup$

You can try this: 

cond = df1['Email'].isin(df2['Email'])
df1.drop(df1[cond].index, inplace = True)

>>df1
    First   Last    Email
2   Joe     Max     [email protected]
3   Will    Bill    [email protected]
Improve this answer
$\endgroup$
1
  • $\begingroup$ This method does not work if you don't have a unique column (email in this case). In this case you can join two or more columns into one with df1[['First', 'Last']].agg('-'.join, axis=1) if they are strings. $\endgroup$
    – alvitawa
    Commented Apr 3, 2022 at 12:47
9
$\begingroup$

Simpler to use isin() with dropna()

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.isin.html

df1[~df1.isin(df2)].dropna()
Improve this answer
$\endgroup$
3
  • $\begingroup$ Quick question, what does the tilde stands for? $\endgroup$
    – Gerardo Jaramillo
    Commented Apr 23, 2021 at 18:23
  • $\begingroup$ It's a negation. Will turn all True to False and vice versa. $\endgroup$
    – Idodo
    Commented Apr 26, 2021 at 8:50
  • 1
    $\begingroup$ This is wrong. It compares the values one at a time, a row can have mixed cases. Even when a row has all true, that doesn't mean that same row exists in the other dataframe, it means the values of this row exist in the columns of the other dataframe but in multiple rows. $\endgroup$
    – anishtain4
    Commented May 13, 2022 at 17:08

Not the answer you're looking for? Browse other questions tagged or ask your own question.