2
$\begingroup$

I have been watching a tutorial on stock price prediction with multivariate linear regression and the tutor replaces missing value data, NaN, with the outlier -99999. Why and how do replacements like that do not skew the data and provide a biased or incorrectly-trained classifier?

$\endgroup$
  • 2
    $\begingroup$ I guess it is not a well-known procedure due to the fact that it will be biased. Finding the mean of missing features is more usual than what you have referred. $\endgroup$ – Media Jul 1 '18 at 15:14
  • 1
    $\begingroup$ Sounds like a really bad idea. Why don't you ask your tutor? $\endgroup$ – Has QUIT--Anony-Mousse Jul 1 '18 at 22:05
2
$\begingroup$

-999999 is pretty common when you work with trees or forests (you imputatte missing values with the value that differs much from other values to perform better splitting).

Maybe (!) in some rare cases imputation by -999999 may be useful (for example, I used it by clustering) though.

Mean/median as advised are really good-working approaches. Including, for example, boolean column that indicates missing values (=1; before imputation) and non-missing (=0).

| improve this answer | |
$\endgroup$
0
$\begingroup$

This process is called imputation, and that is the process of replacing missing data with substituted values. This involves using two methods replacement by mean and replacement by median to substituted the missing values, and these imputation methods is more usual that what you referred, so it is better for you to start from data understanding or why your data are missing etc.

For example, if there is a dataset that have great outliers, I would us imputation by median. In other hand, if your missing values are randomly distributed (or with small size), you might be better to use imputation by mean. If you substitute missing values with means, the mean is preserved.

| improve this answer | |
$\endgroup$
0
$\begingroup$

Maybe , your tutor is just replacing missing values with -99999 for the sake of it.

He/She wants a number to represent the missing values.

I have seen this practice for Well Log Data, where the software cannot deal with NaNs directly and has to have NaNs represented by a number, in this case -99999.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.