5
$\begingroup$

enter image description here

Here is the confusion matrix I got when I was playing with Forest Type Cover Kaggle dataset : Link.

In the matrix, light color and higher numbers represent higher error rates, so as you can see, lots of mis-classification happened between class 1 and 0 .

I wonder what kind of methods I can use to reduce these two error rates though improvements have been made through combing two classifiers, Random Forest and Extra Tree. Will stacking help in this case?

Data can be found on https://www.kaggle.com/c/forest-cover-type-prediction/data

Improve this question
$\endgroup$
1
  • $\begingroup$ The dataset is not skewed and every class has the same amount of training instances. $\endgroup$
    – Chenxiong Yi
    Dec 15, 2017 at 7:13

1 Answer 1

Reset to default
1
$\begingroup$

Welcome to the site!

I think Ensemble Method is very tricky. when one of the model doesn't work well then the accuracy of the Ensemble also goes down.

For instance let us consider that you are using RandomForest(RF) and Rpart for classification and RF accuracy is 90% and Rpart accuracy is 60%. If you ensemble these 2 models then the Ensemble accuracy goes down.

Coming to your scenario, you need to be very careful at the time of stacking, you need to select the models that are performing moderately and then stack them to improve the accuracy.

How is the distribution of 0/1's, if they are imbalanced then you need to balance to improve the accuracy of the model. To handle imbalance data situation we use packages like SMOTE,ROSE etc.

Feature Engineering like adding external factors or adding new features, might help you to improve your models accuracy.

Do let me know if you have any additional questions.

Improve this answer
$\endgroup$
11
  • $\begingroup$ Thank you for your answer! All classes in this data-set actually have same size, so 0 and 1 are not minority in this case. I feel that since 0 and 1 are pretty much the same in almost all features, it is hard to classify them correctly. Do you know any way to handle this scenario? $\endgroup$
    – Chenxiong Yi
    Dec 13, 2017 at 7:27
  • $\begingroup$ So the data is normally distributed, what all features do you have? $\endgroup$
    – Toros91
    Dec 13, 2017 at 7:28
  • $\begingroup$ kaggle.com/c/forest-cover-type-prediction/data you can see all features here. By the way, since all classes have same sizes, shouldn't the distribution be uniform? $\endgroup$
    – Chenxiong Yi
    Dec 13, 2017 at 7:30
  • $\begingroup$ Can you explain the above statement with an example? $\endgroup$
    – Toros91
    Dec 13, 2017 at 7:35
  • $\begingroup$ I just mean no class has more training data than other classes. Sorry for the confusion. $\endgroup$
    – Chenxiong Yi
    Dec 13, 2017 at 7:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.