1
$\begingroup$

I am using Two-Class Boosted Decision Tree to train model.

Evaluation result I'd say really good.

But when I am using real dataset - the result is very bad.

What can possibly go wrong that makes such huge difference?

Below is the screenshot of my model:

enter image description here

Two Class Boosted Decision Tree parameters (default):

enter image description here

$\endgroup$
1
$\begingroup$

Your question is not clear. There's 2 ways to understand it. Which dataset did you use to train your model?

  1. You trained and tested on a premade dataset. The result is great. Then you applied this model to real dataset and the result is really bad.

If this is the case, you should retrain on your real dataset or apply some Transfer Learning techniques to your current model.

  1. You trained and tested on a premade dataset. The result is great. Using the same model, you trained and tested on real dataset but the result is much worse.

I can't tell exactly the reason for this. Normally, real data is much more noisy. Did you handle missing data and do some feature engineering before training?

$\endgroup$
2
  • $\begingroup$ Thanks for your respond. I trained and tested on a premade dataset. The result is great. Then I applied this model to real dataset and the result is really bad. I handle missing data, mostly numeric numbers substituted with 0. I did not do any feature engineering, i think I need to learn more about that. $\endgroup$ – Serdia Feb 24 '18 at 19:58
  • $\begingroup$ I'm sorry to miss your comment before. Replacing missing numeric numbers with 0 might not be a good practice. There's better techniques to handle missing values that's worth trying. $\endgroup$ – TQA Mar 3 '18 at 4:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.