1
$\begingroup$

In the given task, they've provides us with 2 datasets namely the test & train dataset. So, I was wondering if we could merge those 2 datasets into 1 data-frame and delete the duplicates. Would you advise me to follow this approach or is it going to have an adverse effect like overfitting?

$\endgroup$

1 Answer 1

1
$\begingroup$

Test data should never be used in the training process. The reason why there are test data ist that you test your model performance against data the model has never seen before. By doing so, you "simulate" a situation in which the trained model is "in production" so that you can get a good idea of how well it works.

In most cases a model is able to reproduce what it has seen during training. So model performance will usually be "rather good" when you test the model based on data it has seen during trianing. However, showing "new" data (not seen during training, i.e. the "test data") usually gives a good idea how well the model generalizes (with "new" data).

Train your model on the training data only, make a prediction on both data (train/test), and compare both results. I guess you will find different results in terms of model performance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.