I'll explain further, so I'm taking a data science course on cleaning and preparing data and I'm on the how to handle missing data section. So the question is essentially what you see above except it asked how to handle it and gave five options. I selected the best course of action would be to calculate the mean/median whichever you choose of both the test set and training set separately and fillna to the respective set. The course said that this is wrong and you want to calculate the training set and use it to fillna both the train and test sets. Wouldn't this create a bias in your test set causing a higher scored model than you actually achieved? I guess I feel like calculating the central tendencies separately would create a more accurate score even if it made the model seem less impressive. Thanks in advance to any who answer and help me understand I'm very new and would love to understand the reasoning behind it if I'm wrong.

  • $\begingroup$ Could you give a small example? I don't see clearly the different solutions to manage NA cells in the training and in the test sets. $\endgroup$ Jun 25 at 8:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.