New answers tagged

0

You have overfitting when your model corresponds too closely to the training data and may therefore fail to fit additional data or predict future observations reliably. Basically, when the performance on training (or validation) set are much more better than on test set. You have the opposite case: performance on test set much better than validation set. ...


1

I think that the issue depends on what you'd expect the model to learn: If the model is supposed to "know" the users it has seen during training, i.e. exploit the user id in order to infer particular choices for a specific user, then I don't see the point in adding this kind of frequency feature: the model already "knows" what choices ...


1

No, I'm afraid you won't get the best model if you don't ask for it specifically. But don't despair - just set the fit parameter of early_stopping_rounds to a number - and it will stop after this number of rounds in which the validation loss got worse. The number of rounds that would work best for you should be set experimentally (i.e., just fiddle around ...


5

This is a good example of what is usually called "data leakage" -- you are bleeding information from your test set back to your model set. A certain amount of this is inevitable, and it's why (especially for deep learning) problems data scientists often split up data sets into training, validation, and holdout sets. The validation/testing set is ...


5

What you're doing is manual feature selection based on the test set. You're right that it's not correct to proceed this way: in theory, feature selection should be done using only the training set and a validation set, not the final test set. The risk is data leakage: you're modifying the model using information from the test set. Maybe the performance is ...


1

In general, the max depth parameter should be kept at a low value in order to avoid overfitting: if the tree is deep it means that the model creates more rules at a more detailed level using fewer instances. Very often some of these rules are due to chance, i.e. they don't correspond to a real pattern in the data. Overfitting is visible in your graphs from ...


4

As can bee seen in the screenshot, it says the loss is 1/2. Where does that 1/2 is coming from? What does the predictor $h_S(\mathbf{x})$ actually do? In simple English, it predicts the label from the training set if $\mathbf{x}$ is a member of the training set, and otherwise it predicts 0. Obviously, this predictor will perform quite well when evaluated ...


4

To clarify the proposed classifier (because I think it depends on some nonstandard notation): classify any point that is in the (finite) training set according to its true label. (Clearly this gives you an error rate of 0 on the training set.) For any point not in the training set, just predict label 0. Now, consider a random new point in the gray square: ...


Top 50 recent answers are included