1
$\begingroup$

What are the risks if the test data is significantly different from the training data?

Is the most significant problem related to both?

$\endgroup$
  • $\begingroup$ "Is the most significant problem related to both?" What do you mean by related to both? Both data sets? Something else? $\endgroup$ – Trilarion May 23 at 14:39
  • $\begingroup$ the test data is significantly different than the training data @Trilarion $\endgroup$ – lilis gumilang May 25 at 2:00
1
$\begingroup$

The main risk is underfitting, a model trained on a significantly different dataset will poorly predict the test set

$\endgroup$
  • $\begingroup$ okay. thanks for your answer @lorenzo_fattoriale $\endgroup$ – lilis gumilang May 25 at 2:01
0
$\begingroup$

In order for the predictions to be as accurate as possible, the training data should be as representative as possible of the test data. The training data is never going to be exactly accurate, but should be as close as possible. Usually the best way to achieve this is by using a larger training set if possible or using random sampling if not.

If there is a significant difference then the biggest risk is that the model will under fit the test data and will give you inaccurate predictions.

You could also try splitting the training data into training and validation sets to see how the model works on that before applying it to your model and see how much of a difference there is.

$\endgroup$
  • $\begingroup$ yes.. I think it same about prediction but I need more than thought which supports thank you $\endgroup$ – lilis gumilang May 25 at 2:03
0
$\begingroup$

Your model will be way off in terms of prediction accuracy(Underfitting the test dataset ), but that's not an issue because you can either collect more data and fine-tune your model as you miss predictions from unseen and far from what you trained on so you can cover all sorts of inputs in the long run. Or if your test inputs are always unpredictable, and your classes are imbalanced ( cancer detection example ) , train only on the dominant class examples, and label any input that is different from what you trained on as the dominated class.

$\endgroup$
  • $\begingroup$ yes.. thanks for your answer. that's very helpful $\endgroup$ – lilis gumilang May 25 at 2:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.