Skip to main content
12 votes
Accepted

How to use SimpleImputer Class to replace missing values with mean values using Python?

Your error is due to using Simple Imputer's fit and fit_transform on a numpy array. Here's how i used it on a Dataframe ...
Blenz's user avatar
  • 2,084
9 votes

How to replace NA values with another value in factors in R?

You can use this function : forcats::fct_explicit_na library(forcats) fct_explicit_na(DF$col, na_level = "None") Usage It can be used within the mutate ...
eg-r's user avatar
  • 176
9 votes
Accepted

How to impute Missing values not the usual way?

To decide which strategy is appropriate, it is important to investigate the mechanism that led to the missing values to find out whether the missing data is missing completely at random (MCAR), ...
Noah Weber's user avatar
  • 5,699
7 votes

How do GBM algorithms handle missing data?

LIGHTGBM will ignore missing values during a split, then allocate them to whichever side reduces the loss the most. https://github.com/microsoft/LightGBM/issues/2921 There are some options you can set ...
Noah Weber's user avatar
  • 5,699
7 votes
Accepted

How to impute missing text data?

First most of the time there's no "missing text", there's an empty string (0 sentences, 0 words) and this is a valid text value. The distinction is important, because the former usually ...
Erwan's user avatar
  • 25.5k
6 votes
Accepted

How to handle missing data for machine learning

There are three main approaches to handling missing data. Impute - use some method to fill in the missing values with reasonable guesses. You could interpolate between two time points, take the ...
Nuclear Hoagie's user avatar
6 votes
Accepted

How to replace NA values with another value in factors in R?

You need to add "None" to the factor level and refactor the column DF$col. I added an example script using the iris dataset. ...
phiver's user avatar
  • 718
6 votes
Accepted

Predicting Missing Features

A simple approach could be the following: suppose $i \in \{0,1\}^d$ is the vector you want to predict which of the $0$ entries could be $1$ and $j \in J$ the rest of the feature vectors. Take the $k$ ...
Dani Mesejo's user avatar
  • 2,226
6 votes

What predictive model to use to impute Gender?

You can use one of the popular gradient boosting tree implementations such LightGBM and XGBoost as your predictive model. They can handle missing values during training and the results are often ...
Simon Larsson's user avatar
6 votes

How to impute Missing values not the usual way?

A trick I have seen on Kaggle. Step 1: replace NAN with the mean or the median. The mean, if the data is normally distributed, otherwise the median. In my case, I have NANs in Age. Step 2: Add a ...
FrancoSwiss's user avatar
  • 1,077
6 votes

How to handle missing value if imputation doesnt make sense

I think this is a good solution. You could also try to set a unique negative value for non-married people, especially if you are using a tree-based model.
Just trying's user avatar
6 votes

How to handle missing value if imputation doesnt make sense

You could consider setting years_married to -1, then it is different from columns for the ones that are just married and could thus be understood by a decision tree....
justinlk's user avatar
  • 136
5 votes
Accepted

What predictive model to use to impute Gender?

I agree with Simon's advice. I find that the gains that you obtain from using any external method of imputation is often inferior to an internal method, and on top of this, exposes you to even more ...
aranglol's user avatar
  • 2,196
5 votes

How to use SimpleImputer Class to replace missing values with mean values using Python?

SimpleImputer also works fine. ...
Sharad Singh's user avatar
5 votes

When to use missing data imputation in the data analysis problem?

Generally speaking you have two options: impute the missing data discard the missing data Due to the fact that ML models perform better with more data, the former is usually preferred. However, you ...
ILM91's user avatar
  • 338
5 votes

Imputation of missing values based on target variable

You can absolutely do this. Whether it's optimal depends on the missingness mechanism. If the missing values in this column are independent of other columns, then this is probably the best possible ...
Ben Reiniger's user avatar
  • 11.9k
4 votes
Accepted

Scikit Learn Missing Data - Categorical values

We would need more information on the prediction problem and the features to be able to give something more precise. Anyhow, I am surprised no answer so far included all possible options since they ...
Ricardo Cruz's user avatar
  • 3,420
4 votes

Naive Bayes Should generate prediction given missing features (scikit learn)

Your question is sensible. The way in which posterior probability is calculated in the classical Naive Bayes classifier (in sklearn) is like summation of the conditional probabilities of the all the ...
Arun Aniyan's user avatar
4 votes
Accepted

Missing Values in Data

Various methods are available for fill missing values in data. Ignore the tuple is the simplest and not effective method. Fill the missing value manually. Use a global constant to fill the missing ...
SWATHY M NAIR's user avatar
4 votes
Accepted

Data that's not missing is called...?

Depends on content, but I would probably go for "observed" (vs. "unobserved"). A suitable direct antonym of "missing" might be "extant".
R Hill's user avatar
  • 1,105
4 votes
Accepted

Missing outputs in multiple-output neural net

Since we are talking about multiple different types of targets (classes versus numerical for example) we already need a composite loss function. I will consider how to balance the different composite ...
Jan van der Vegt's user avatar
4 votes

Fill missing values AND normalise

I don't understand why you would like to fill values with zeros ! This would basically mean, "this guy, who is 170 cm tall, weights 0 kg" and would fool your network. In my opinion, you have two ...
Robin's user avatar
  • 1,337
4 votes
Accepted

Dealing with NaN (missing) values for Logistic Regression- Best practices?

Use caution when removing features with missing values. Sometimes the fact that a feature has missing values is valuable data in and of itself. What you are asking about is called imputation. A ...
bstrain's user avatar
  • 421
4 votes

Data prepration for logistic regression : Value either "not available" or a "year"

First use a binary 0 (no renovation) and 1 (renovation) which works perfect with logistic regression. Using the exact date is a bad practice. It guides the model in the direction of over-fitting on ...
Esmailian's user avatar
  • 9,342
4 votes

How to fill missing values by looking at another row with same value in one column(or more)?

Sorted and did a forward-fill NaN ...
10xAI's user avatar
  • 5,634
3 votes

How to handle missing data for machine learning

Interpolation seems like it would make sense in this case: any time you miss a day, take an average of the before and after. As an aside, I don't think you have to give up on the missing weather ...
CalZ's user avatar
  • 1,663
3 votes

Missing Values in Data

Yes there are so many approaches to handle missing data or missing values depending on the task and the property of the data itself. For example in time series you can think about forward filling or ...
HatemB's user avatar
  • 326
3 votes
Accepted

How can I handle missing categorical data that has significance?

Imputation and dealing with missing data a broad subject; you should start by researching standard material on this subject. The first question to figure out is Why is some data missing? and What is ...
D.W.'s user avatar
  • 3,381
3 votes
Accepted

Replacing missing value by class conditional mean

Your intuition about 'no effect' is true in some sense. But this replacement may be not the best use of the information you have. The choice of missing value treatment depends on your initial ...
David Dale's user avatar
  • 1,551
3 votes

Missing Values in Data

First of all, if most of your data is missing, you are in trouble anyway. You need to ask why is most of the data missing, and also, why are the data you observed not missing. Being missing is very ...
astaines's user avatar
  • 131

Only top scored, non community-wiki answers of a minimum length are eligible