Questions tagged [ensemble-modeling]

In machine learning, ensemble methods combine multiple algorithms to make a prediction. Bagging, boosting, and stacking, are some examples.

Filter by
Sorted by
Tagged with
3
votes
1answer
2k views

Algorithmic approach to model blending

Model blending -- by which I mean creating multiple sets of predictions from models that have the same dependent variable and the same or similar independent variable candidates, as opposed to model ...
5
votes
1answer
170 views

2 stage ensemble -- CV MSE valid in 1st stage but not in 2nd

I'm trying out a Kaggle competition, which puts me in the unusual position of being able to get feedback on my models' "true" performance (you can submit several predictions per day and they give you ...
8
votes
1answer
4k views

what is the difference between "fully developed decision trees" and "shallow decision trees"?

As reading Ensemble methods on scikit-learn docs, it says that bagging methods work best with strong and complex models (e.g., fully developed decision trees), in contrast with boosting methods ...
1
vote
1answer
180 views

Stacked features not helping

I am wondering why my stacked features do not help me to improve against my loss metric. Here's what I'm doing: I am adding new features which are simple the predictions originated from train, predict ...
1
vote
1answer
215 views

Importance of variables in RandomForest in R

I'm using varImpPlot() to evaluate the importance of variables from my rf model, and I can't decide wich variable I have to delete from my model. I'm not sure about the diference between "...
2
votes
1answer
696 views

Pick a model from multiple models using a decision tree

Let us say, I have 4 classification models on a training data set of various examples. Now, I want to choose which 1 out 4 models (or what combination of the 4 ...
6
votes
1answer
10k views

Assumptions/Limitations of Random Forest Models

What are the general assumptions of a Random Forest Model? I could not find by searching online. For example, in a linear regression model, limitations/assumptions are: It may not work well when ...
4
votes
2answers
1k views

How to ensemble classifier incorporating all features in python?

I am doing a text classification task(5000 essays evenly distributed by 10 labels). I explored LinearSVC and got an accuracy of 80%. Now I guess whether accuracy ...
7
votes
2answers
215 views

Why isn't dimension sampling used with gradient boosting machines (GBM)?

GBMs, like random forests, build each tree on a different sample of the dataset and hence, going by the spirit of ensemble models, produce higher accuracies. However, I have not seen GBM being used ...

1 2 3
4