I have two columns in a dataframe namely 'origin' and 'destination' which contains the names of different places. I need to remove the rows that don't contain 'PHX', 'JFK', 'NTU' in either of these columns. If atleast one among the 2 columns has one of these 3 places, the row can stay, else the row must be dropped altogether. Can you please help me to code this part ?

  • $\begingroup$ Which programming language? Please specify it and add a tag. $\endgroup$
    – Spacedman
    Feb 26 '19 at 22:52

This snippet should do the work:

city = ['PHX', 'JFK', 'NTU'] 
colomn_to_exclude = df.apply(lambda row: (row['origin'] not in city) and (row['destination'] not in city))
new_df = df[~colomn_to_exclude]

the second line checks the lines where your exclusion condition is verified and the line after subset the dataframe accordingly.

You could also write something like

colomn_to_exclude = df.apply(lambda row: (row['origin'] not in city) and   (row['destination'] not in city))
df.drop(~colomn_to_exclude, axis=1, inplace=True)

and you won't have to make a copy of your dataframe

  • $\begingroup$ Hi @Robin, I get the following error on trying out your first snippet, the error goes as follows : Type error: set expected at most 1 arguments, got 3 $\endgroup$
    – VishwaV
    Jan 27 '19 at 20:33
  • $\begingroup$ I changed the definition of city in the first line this should work now $\endgroup$ Jan 27 '19 at 20:45
  • $\begingroup$ I have a type error now stating that - '<' not supported between instances of 'numpy.ndarray' and 'str' $\endgroup$
    – VishwaV
    Jan 27 '19 at 21:01
  • $\begingroup$ Can you provide the data frame y $\endgroup$ Jan 27 '19 at 21:08
  • $\begingroup$ It's actually too huge to type it out. It has 48 columns and some lakh rows. I actually posted a minimalistic version of the dataframe with the required conditions alone. Will you need the full dataframe as such ? 🤔 $\endgroup$
    – VishwaV
    Jan 27 '19 at 21:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.