How to delete entire row if values in a column are NaN [closed]

Question

I'd like to drop all the rows containing a NaN values pertaining to a column. Lets assume I have a dataset like this:

Age  Height  Weight  Gender
12    5'7     NaN     M
NaN   5'8     160     M
32    5'5     165     NaN
21    NaN     155     F
55    5'10    170     NaN

I want to remove all the rows where 'Gender' has NaN values. The output i'd like:

Age  Height  Weight Gender
12   5'7     NaN     M
NaN  5'8     160     M
21   NaN     155     F

Thanks in advance!

Answer 1

Well if the dataset is not too large I would suggest using pandas to clean the data. So you would need to first do

Python2

python2 -m pip install pandas

Python3

python3 -m pip install pandas

If you already have anaconda installed you can skip the above step. Next you could go through an IDE (like jupyter) or through the shell type the following commands

import pandas as pd
df = pd.read_csv("filename", dtype=str)
#or if excel file
#df = pd.read_excel("filename", dtype=str)

df = df[pd.notnull(df['Gender'])]

Then you would want to save your result in a file with

df.to_csv("newfile"); 

Answer 2

If you are working on Python Dataframe, Please try this.

import pandas as pd
df = df[np.isfinite(df['Gender'])]

Or you can use notnull instead of isfinite.

import pandas as pd
df = df[pd.notnull(df['Gender'])]