I have two columns in a dataframe namely 'origin' and 'destination' which contains the names of different places. I need to remove the rows that don't contain 'PHX', 'JFK', 'NTU' in either of these columns. If atleast one among the 2 columns has one of these 3 places, the row can stay, else the row must be dropped altogether. Can you please help me to code this part ?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ Which programming language? Please specify it and add a tag. $\endgroup$– SpacedmanFeb 26, 2019 at 22:52
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
5
This snippet should do the work:
city = ['PHX', 'JFK', 'NTU']
colomn_to_exclude = df.apply(lambda row: (row['origin'] not in city) and (row['destination'] not in city))
new_df = df[~colomn_to_exclude]
the second line checks the lines where your exclusion condition is verified and the line after subset the dataframe accordingly.
You could also write something like
colomn_to_exclude = df.apply(lambda row: (row['origin'] not in city) and (row['destination'] not in city))
df.drop(~colomn_to_exclude, axis=1, inplace=True)
and you won't have to make a copy of your dataframe
-
$\begingroup$ Hi @Robin, I get the following error on trying out your first snippet, the error goes as follows : Type error: set expected at most 1 arguments, got 3 $\endgroup$– VishwaVJan 27, 2019 at 20:33
-
$\begingroup$ I changed the definition of city in the first line this should work now $\endgroup$ Jan 27, 2019 at 20:45
-
$\begingroup$ I have a type error now stating that - '<' not supported between instances of 'numpy.ndarray' and 'str' $\endgroup$– VishwaVJan 27, 2019 at 21:01
-
-
$\begingroup$ It's actually too huge to type it out. It has 48 columns and some lakh rows. I actually posted a minimalistic version of the dataframe with the required conditions alone. Will you need the full dataframe as such ? 🤔 $\endgroup$– VishwaVJan 27, 2019 at 21:23