2
$\begingroup$

I am working on a data set and there is an interesting column with missing values, but I don't want to discard the rows (so as not to lose data from other columns) or do imputation (so as not to change the data). Can I work with the dataframe with a column with missing values during exploratory data analysis and only take the slides with no values missing when plotting something with this specific column?

$\endgroup$
0
$\begingroup$

If the number of rows includes missing values are very small according to sample size I recommend dismissing it. But if you decide to keep them according to not lose any information, you can do a bunch of things according to the feature that involves a null value.

You should understand the pattern of the feature column well before deciding the filling method below.

  • You can change the null value as;
    • mean of the column
    • median of the column
    • same as above or below
    • just zero
    • most repeated value along the column
    • etc.

If there is any categorical feature, you can group by a feature like gender and can do the same thing as above. For example, if there is a NaN height value for a male you can fill it with the mean of the heights of males etc.

Besides all you can decide to discard the whole column with:

  • Checking the correlation between the column and the dependent variable
  • Checking the column's representation level of source data with PCA
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.