3
$\begingroup$

Recently, I tested two methods after embedding in my data, using Keras.

  1. Convolution after embedding
  2. Maxpooling after embedding

The first method's loss and validation loss are like, enter image description here

The second method's loss and validation loss are enter image description here

As you can see, the first one reduces the loss a lot for the training data, but the loss increases significantly in the validation set.

The second method's loss for the training data is higher than the first method, but both loss in the training data and validation data are almost same.

Which method is more suitable for further analysis with additional variables?

$\endgroup$

1 Answer 1

8
$\begingroup$

The performance on in-sample data almost does not count. The performance on out-of-sample data is more indicative of how you should expect your model to perform on future inputs.

The second model has better out-of-sample performance. With just that information, I would prefer the second model.

$\endgroup$
1
  • $\begingroup$ I agree the out-of-sample performance is the better measure of a model, but it sounds like the OP doesn't actually want to use either of these models, and instead wants to build a new model with additional variables using one of the two methods. I'm not sure the out-of-sample performance for either method trained with this set of variables is necessarily indicative of which model will have better out-of-sample performance with a different set of variables. If you add many new variables you effectively have an entirely new problem which has little connection to the validation loss seen here. $\endgroup$ Commented Jan 10, 2022 at 14:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.