1
$\begingroup$

I have a question about data cleaning. I am a novice and have just started learning in this field so please pardon my ignorance. Suppose there are two columns and based on some samples taken from both the columns you find the correlation coefficient to be high. Now for the values that aren't there, can you use linear regression to predict or find them out, by using the values you know as training data?

$\endgroup$
  • 1
    $\begingroup$ Yes, as explained by Donald. Also, check - KNNImputer $\endgroup$ – 10xAI Jun 29 at 3:42
1
$\begingroup$

Hi Soumyadeep and welcome to Data Science/Stack Exchange

What you are describing is called regression imputation, and it is a valid method to use on missing data. However, if the data is sparse (lots of missing values), this issue will be more difficult to handle.

In general, missing data can be handled in several ways (row deletion, imputation, substitution, etc). Regression imputation can be used if you have little or no knowledge about the data, but usually it is better to use another method. If you have some domain knowledge about the missing values, like you have an idea what the value should be, usually you can use that knowledge to fill in the missing values. Try some different methods and see which one works best.

A person pointed out that I should check for multicollinearity if the features are both independent. Does it basically mean that one feature is falling in the span of the other feature?

Definition of multicollinearity: There exist one or more exact linear relationships among some of the variables

enter image description here

enter image description here

References: https://en.wikipedia.org/wiki/Multicollinearity https://stats.stackexchange.com/questions/234870/is-multicollinearity-the-issue-here

| improve this answer | |
$\endgroup$
  • $\begingroup$ Hello Donald, thank you for the answer. Greetings! Now I have a follow up question. A person pointed out that I should check for multicollinearity if the features are both independent. Does it basically mean that one feature is falling in the span of the other feature? $\endgroup$ – Soumyadeep Mukhopadhyay Jul 22 at 0:32
  • $\begingroup$ For the independent features, you should check for multicollinearity of the 2 columns you mention in your question. This does not necessarily mean the features will fall in the span (think you are referring to range of values) of the other feature. High multicollinearity means multiple features are highly correlated to or dependent on each other. $\endgroup$ – Donald S Jul 22 at 4:20
  • $\begingroup$ More importantly, for imputation, you want a high correlation so the imputation can be more accurate. However, when you build your model, you want to use some L1 or L2 regularization to counteract the issues that this multicollinearity or correlation will cause. If the correlation is high, you may need a higher regularization factor to avoid overfitting. Many models will include regularization(s) internally, so you only need to adjust the value of the regularization factors. $\endgroup$ – Donald S Jul 22 at 4:20
  • $\begingroup$ Thank you for your answer, again! Given my limited knowledge I am yet to completely understand the idea that you are trying to express. I am glad that you helped me out in this regards. I guess I was under the impression that multicollinearity was somewhat analogous to the vectors being coplanar. $\endgroup$ – Soumyadeep Mukhopadhyay Jul 22 at 15:32
  • $\begingroup$ I added some feedback in my answer above about your comment on multicollinarity and if vectors are coplanar - they should be collinear. $\endgroup$ – Donald S Jul 23 at 1:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.