Questions tagged [ensemble-modeling]

In machine learning, ensemble methods combine multiple algorithms to make a prediction. Bagging, boosting, and stacking, are some examples.

Filter by
Sorted by
Tagged with
4
votes
1answer
947 views

Geometric and harmonic means in ensembling methods

When using ensembling methods for regression, a common approach is to average (using the arithmetic mean) the outputs of the weak learners in order to obtain the output of the ensemble. Is there a ...
1
vote
1answer
9k views

How to avoid memory error with Pandas pd.read_csv method call with GridSearchCV usage for DecisionTreeRegressor model?

I have been implementing a DecisionTreeRegressor model in Anaconda environment with a data set sourced from a 20 million row, 12-dimensional CSV file. I could get the chunks off of the data set with ...
6
votes
3answers
243 views

What are the individual models within a machine learning ensemble called?

I am aware that an ensemble machine learning model is a stack of two or more machine learning models. Is there a word to refer to those individual models that go into the ensemble model? (i.e. a ...
1
vote
2answers
255 views

Questions on ensemble technique in machine learning

I am studying the ensemble machine learning and when I read some articles online, I encountered 2 questions. 1. In this article, it mentions Instead, model 2 may have a better overall performance ...
1
vote
1answer
174 views

Multi-dimensional regression with ensemble models?

I'm curious to know whether boosting, random forests or other types of ensemble models can perform multi-dimensional regression. To be precise: That means multiple outputs (multi-dimensional labels) ...
1
vote
0answers
59 views

AdaBoost - Ensemble model perform poor than best weak classifier

Can Adaboost's ensemble classifier perform worse than the best of the weak learners considered? If so when in what case of weak learner the ensemble learning does not perform better?
2
votes
1answer
30 views

How to approach model reporting task

I have been tasked to report on an ensemble model that was created in h2o which includes several model subtypes such as Random Forest, GBM, linear models etc. The end goal is to predict churn rates ...
1
vote
3answers
223 views

Retrieve user features in real time from UserId for prediction

Let's say I'm building an app like Uber and I want to predict the user's most likely destination based on the user's past history, current latitude/longitude, and time/date. Here is the proposed ...
1
vote
1answer
278 views

testing new data in model

I have ensembled 3 algorithms as below, ...
2
votes
2answers
266 views

Combining Different Models

I have N models, each of them are used to predict on a set of data. I am currently combining their predictions by averaging across rows. Need suggestions on combining their predictions. End goal is to ...
1
vote
1answer
26 views

Re: Missing Value

I have a large dataset that has session length records per user basis. And I am trying to predict the purchase behaviour based on the session length. But this data has multiple zero in the session ...
2
votes
2answers
972 views

Taking average of multiple neural networks?

I'm fitting a neural network using a very small data set, so try splitting the data into training and validation sets. (there is a separate test set) If I split training/validation randomly multiple ...
5
votes
3answers
265 views

How to create an ensemble that gives precedence to a specific classifier

Suppose that in a binary classification task, I have separate classifiers A, B, and C. If I ...
0
votes
3answers
209 views

Ensemble Model vs Normal model

If I get 95+ % accuracy in normal models, should I still consider Ensemble models? Why should I choose Ensemble models over normal models?
-1
votes
1answer
136 views

Ensemble Probabilities of the different models

I have a Multiclass Dataset and I am getting probabilities of classes from RandomForest. However, I want to divide the dataset for each class as examples of either the case belongs to that class or ...
-1
votes
1answer
72 views

How to average classifiers with AUC metric?

I am modeling a binary classification and my loss function is the gini function (normalized area under the curve). Here's my implementation: Split the data with k-folds Train k classifiers Now I ...
1
vote
2answers
78 views

Learning Algorithm that decide which model gives better results for each testing instance

Is their any existing Ensemble technique which uses subset of training data to predict which algorithm is better for predicting each instance of testing data? Let's say we have N sized training set ...
4
votes
0answers
2k views

How to tune weights in Voting Classifier (Sklearn)

I am trying to do the following: ...
4
votes
1answer
3k views

Sklearn Aggregating Multiple Fitted Models Into A Single Model? (binary classification)

My problem context: dataset too big to fit into memory. binary classification [0,1] 30 csv files in a directory with exactly 30,000 samples (rows) each file contains 15,000 ...
1
vote
1answer
3k views

How can I do tree_method ='exact' in XGBoost classifier?

I am doing XGBoost classification on a huge data set and its showing: ...
2
votes
2answers
379 views

How to measure the correlation of different algorithms

In stacked generalization, several algorithms are trained on the training set (i.e. at layer 1) and their predictions are then stacked using a layer 2 model. In many documentations, it is said that it ...
0
votes
1answer
53 views

What kind of algorithms can be used as a stacker in stacked generalization?

In stacked generalization, several algorithms (I use some random trees, booster trees, etc.) are first trained and used to make the predictions which are used as input for another algorithm. However, ...
3
votes
1answer
94 views

Why some people add results from PCA and other dimensional reductions techniques as features

I often see in a well-known datascience competition platform, that a lot of people apply some dimension reductions techniques, but instead of using it to reduce the number of features (complexity) of ...
6
votes
1answer
220 views

How predictions of level 1 models become training set of a new model in stacked generalization.

In stacked generalization, if I understood well, we divide the training set into train/test set. We use train set to train M models, and make predictions on test set. Then we use the predictions as ...
1
vote
2answers
115 views

How to use an existing model as in input into a new model

We have a click-model which is currently being used for search ranking in production, and I want to create a new model which takes the old model click probability as one input and adds some other ...
0
votes
2answers
329 views

Ensemble learning [closed]

I am currently working to build a mathematical model to predict the stock market. I learned that the best way to do such thing is no longer to make one big best model, but rather to gather several ...
3
votes
2answers
1k views

Improving classifier performances in R for imbalanced dataset

I have used an "adabag"(boosting + bagging) model on an imbalanced dataset (6% positive), I have tried to maximized the sensitivity while keeping the accuracy above 70% and the best results I got were:...
0
votes
1answer
3k views

Ensemble modelling using model's probabilities

In a classification project, on the training sets, I ran a selection of classifiers. These give me about 20-30% accuracy at best. For each sample, I generate probabilities of each class. I want to ...
2
votes
2answers
587 views

Is there an R package for Locally Interpretable Model Agnostic Explanations?

One of the researchers, Marco Ribeiro, who developed this method of explaining how black box models make their decisions has developed a Python implementation of the algorithm available through Github,...
2
votes
2answers
77 views

How to do boosting in model-ensembling?

Boosting is a sequential technique in which, the first algorithm is trained on the entire dataset and the subsequent algorithms are built by fitting the residuals of the first algorithm, thus giving ...
7
votes
5answers
2k views

Does ensemble (bagging, boosting, stacking, etc) always at least increase performance?

Ensembling is getting more and more popular. I understand that there are in general three big fields of ensembling, bagging, boosting and stacking. My question is that does the ensembling always at ...
60
votes
5answers
86k views

GBM vs XGBOOST? Key differences?

I am trying to understand the key differences between GBM and XGBOOST. I tried to google it, but could not find any good answers explaining the differences between the two algorithms and why xgboost ...
2
votes
1answer
96 views

Decision Tree Ensembling

In order to ensemble a decision tree, let me explain the specific situation. I have split the dataset in 5 sections, per protocol e.g. TCP, HTTP etc. Now I've trained a decision tree for each one and ...
1
vote
1answer
878 views

How to use ensemble of models in FM or FFM?

I am using Factorization Machines ( libfm) and also the Field Aware Factorization Machines (libffm) for a kaggle competition. I am currently using the single models of each respectively for ...
1
vote
0answers
490 views

Compare probability estimates of two classifiers

Consider two binary classifiers A and B. Suppose that both A and B are predicting the same target, but that A is trained on a subset of the data for which a different set of features is available than ...
3
votes
1answer
215 views

Ensemble Techniques for multilabel data

I observed that Adaboost or Bagging ensemble classifiers present in sklearn only work for single label training data. How do I use these for multilabel data?
1
vote
2answers
89 views

Predictive Modeling of Multiple Items

I have a dataset of Social Media Post and want to predict the number of "thumbs up" it will receive over time. ...
6
votes
2answers
28k views

What does Negative Log Likelihood mean?

I have a data set which has continuous independent variables and a continuous dependent variable. To predict the dependent variable using the independent variables, I've run an ensemble of regression ...
2
votes
1answer
171 views

Where does the random in Random Forests come from?

As the title says: Where does the random in Random Forests come from?
3
votes
1answer
511 views

Is SuperLearning actually different to stacking,or are they essentially the same thing?

Articles which use the terms 'stacking' and 'Super Learner' often seem to use the terms interchangeably. Is the Super Learner algorithm a specific form of the more generic stacking concept, or is ...
1
vote
1answer
57 views

how to choose classifer

is the best way to create the most accurate classifier to train a bunch of classifying algorithms like ANN, SVM, KNN, etc, and test different parameters to get optimal parameters for each classifier, ...
1
vote
1answer
476 views

Variable importance in ensemble models

I have noticed that when you make a small decision tree model, and then extend the model by creating an ensemble of trees around the same tree settings, the variable importance is diluted in the sense ...
2
votes
1answer
2k views

EasyEnsemble explaination

Could someone please explain how the EasyEnsemble algorithm works? Im using it for a prediction model for imbalanced minority class. Please don't refer me to this paper, as it makes no sense to me. ...
4
votes
2answers
140 views

clustering plus linear model versus non linear (tree) model

a team has to create models that predict the cost of deploying a machine over time. This is a regression problem. The team is further divided into two groups, A and B. Group A puts lots of ...
4
votes
3answers
2k views

Ensembling vs clustering in machine learning

Following the raise of ensembling (e.g ensembling of xgboost learners) after its recurrent wins in Kaggle competitions, using it has become a tradition in machine learning. However, some argue that ...
3
votes
1answer
2k views

Algorithmic approach to model blending

Model blending -- by which I mean creating multiple sets of predictions from models that have the same dependent variable and the same or similar independent variable candidates, as opposed to model ...
5
votes
1answer
170 views

2 stage ensemble — CV MSE valid in 1st stage but not in 2nd

I'm trying out a Kaggle competition, which puts me in the unusual position of being able to get feedback on my models' "true" performance (you can submit several predictions per day and they give you ...
8
votes
1answer
4k views

what is the difference between “fully developed decision trees” and “shallow decision trees”?

As reading Ensemble methods on scikit-learn docs, it says that bagging methods work best with strong and complex models (e.g., fully developed decision trees), in contrast with boosting methods ...
1
vote
1answer
175 views

Stacked features not helping

I am wondering why my stacked features do not help me to improve against my loss metric. Here's what I'm doing: I am adding new features which are simple the predictions originated from train, predict ...
1
vote
1answer
215 views

Importance of variables in RandomForest in R

I'm using varImpPlot() to evaluate the importance of variables from my rf model, and I can't decide wich variable I have to delete from my model. I'm not sure about the diference between "...