3
$\begingroup$

I am working on a data set and there is an interesting column with missing values, but I don't want to discard the rows (so as not to lose data from other columns) or do imputation (so as not to change the data). Can I work with the dataframe with a column with missing values during exploratory data analysis and only take the slides with no values missing when plotting something with this specific column?

$\endgroup$
1
$\begingroup$

I believe you are looking to work along with the missing values in particular column(X) where column(W,Y,Z) have important values in these rows and can't discard or do imputation, especially for plotting them visually.

Yes its possible, considering:

  1. When you only plan to plot other columns(W,Y,Z excluding column X) to view them visually

  2. When you only plan to include column (X) in EDA, there is a python package missingno that deals with data visualization for missing values.

    Here's the python package link

    Click here for missingno youtube demo

$\endgroup$
1
  • $\begingroup$ @josé-augusto if the answer has solved your question please consider accepting it by clicking the check-mark . This indicates to the wider community that you've found a solution and gives some reputation to both the answerer and yourself. Kindly follow stackoverflow norms $\endgroup$ Oct 12 at 4:53
0
$\begingroup$

If the number of rows includes missing values are very small according to sample size I recommend dismissing it. But if you decide to keep them according to not lose any information, you can do a bunch of things according to the feature that involves a null value.

You should understand the pattern of the feature column well before deciding the filling method below.

  • You can change the null value as;
    • mean of the column
    • median of the column
    • same as above or below
    • just zero
    • most repeated value along the column
    • etc.

If there is any categorical feature, you can group by a feature like gender and can do the same thing as above. For example, if there is a NaN height value for a male you can fill it with the mean of the heights of males etc.

Besides all you can decide to discard the whole column with:

  • Checking the correlation between the column and the dependent variable
  • Checking the column's representation level of source data with PCA
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.