25
$\begingroup$

I would like to compare one column of a df with other df's. The columns are names and last names. I'd like to check if a person in one data frame is in another one.

$\endgroup$
  • $\begingroup$ Could you please indicate how you want the result to look like? Is it a df with names appearing in both dfs, and whether you also need anything else such as count, or matching column in df2 ,etc. Thanks! $\endgroup$ – The Lyrist Jun 12 '18 at 22:39
  • $\begingroup$ check out pandas.pydata.org/pandas-docs/stable/generated/… $\endgroup$ – oW_ Jun 12 '18 at 22:56
  • $\begingroup$ You could inner join the two data frames on the columns you care about and check if the number of rows in the result is positive. $\endgroup$ – dsaxton Jul 13 '18 at 13:41
  • $\begingroup$ FYI, comparing on first and last name on any decently large set of names will end up with pain - lots of people have the same name! $\endgroup$ – Ken Syme Jul 13 '18 at 20:31
24
$\begingroup$

If you want to check equals values on a certain column let's say Name you can merge both Dataframes to a new one:

mergedStuff = pd.merge(df1, df2, on=['Name'], how='inner')
mergedStuff.head()

I think this is more efficient and faster then whereif you have a big data set

$\endgroup$
  • 1
    $\begingroup$ I think we want to use an inner join here and then check its shape. $\endgroup$ – dsaxton Jul 13 '18 at 13:43
11
$\begingroup$
df1.where(df1.values==df2.values).notna()

True entries show common elements. This also reveals the position of the common elements, unlike the solution with merge.

$\endgroup$
  • $\begingroup$ what is df. in your answer? There are only df1 and df2 but no df $\endgroup$ – LearneR Jul 24 '19 at 10:43
4
$\begingroup$

Comparing values in two different columns

Using set, get unique values in each column. The intersection of these two sets will provide the unique values in both the columns.

Example:

df1 = pd.DataFrame({'c1': [1, 4, 7], 'c2': [2, 5, 1], 'c3': [3, 1, 1]}) df2 = pd.DataFrame({'c4': [1, 4, 7], 'c2': [3, 5, 2], 'c3': [3, 7, 5]}) set(df1['c2']).intersection(set(df2['c2']))

Output: {2, 5}


Comparing column names of two dataframes

Incase you are trying to compare the column names of two dataframes:

If df1 and df2 are the two dataframes: set(df1.columns).intersection(set(df2.columns))

This will provide the unique column names which are contained in both the dataframes.

Example:

df1 = pd.DataFrame({'c1': [1, 4, 7], 'c2': [2, 5, 1], 'c3': [3, 1, 1]})
df2 = pd.DataFrame({'c4': [1, 4, 7], 'c2': [3, 5, 2], 'c3': [3, 7, 5]})

set(df1.columns).intersection(set(df2.columns))

Output: {'c2', 'c3'}

$\endgroup$
  • 1
    $\begingroup$ I think the the question is about comparing the values in two different columns in different dataframes as question person wants to check if a person in one data frame is in another one. $\endgroup$ – Divyanshu Shekhar Jun 13 '18 at 7:04
  • 1
    $\begingroup$ Thanks, I got the question wrong. I've updated the answer now. $\endgroup$ – aathiraks Jun 13 '18 at 9:24
2
$\begingroup$

You can double check the exact number of common and different positions between two df by using isin and value_counts()

Like that:

df['your_column_name'].isin(df2['your_column_name']).value_counts()

Result:

example isin

True = common False = different

$\endgroup$
  • $\begingroup$ This should be the answer in my opinion. $\endgroup$ – Superdooperhero Jan 16 at 9:12
0
$\begingroup$

Note that the columns of dataframes are data series. So if you take two columns as pandas series, you may compare them just like you would do with numpy arrays.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.