Questions tagged [ensemble-modeling]

In machine learning, ensemble methods combine multiple algorithms to make a prediction. Bagging, boosting, and stacking, are some examples.

Filter by
Sorted by
Tagged with
0
votes
0answers
23 views

How does XGBoost perform in Parallel

So what I know about boosting technique, Like we train the data and update the weights of falsely predicted values or try to minimize the loss in the next model. So basically, it's the sequential ...
1
vote
1answer
513 views

checking model stability - Performance for different class

I tried to do multi-class classification problem. The goal is to predict whether the match will be won by HomeTeam, AwayTeam or Draw. I did feature engineering from the attributes and finally came up ...
2
votes
1answer
1k views

Training an ensemble of small neural networks efficiently in TensorFlow 2

I have a bunch of small neural networks (say, 5 to 50 feed-forward neural networks with only two hidden layers with 10-100 neurons each), which differ only in the weight initialization. I want to ...
3
votes
3answers
57 views

Understanding Weighted learning in Ensemble Classifiers

I'm currently studying Boosting techniques in Machine Learning and I happened to understand that in Algorithms like Adaboost, each of the training samples is given a weight depending on whether it was ...
1
vote
1answer
36 views

Boosting algorithms only built with decision trees? why?

My understanding of boosting is just training models sequentially and learning from its previous mistakes. Can boosting algorithms be built with bunch of logistic regression? or logistic regression + ...
4
votes
3answers
428 views

Is it possible to build ensemble models without a decision tree?

Is it possible to build ensemble models without a decision tree? I know that the description of ensembles itself suggests otherwise. However, I am really new to machine learning and all the ensemble ...
0
votes
1answer
11 views

Additional business rules in ensemble methods (RF, Boosted Trees)

How is it possible (if at all) to implement additional business constraints to an ensemble machine learning model, such as random forests or boosted trees? These additional business rules can be ...
0
votes
0answers
10 views

How can I use Ensemble learning of two models with different features as an input?

I have a fake news detection problem and it predicts the binary labels "1"&"0" by vectorizing the 'tweet' column, I use three different models for detection but I want to use ...
1
vote
1answer
9k views

How to avoid memory error with Pandas pd.read_csv method call with GridSearchCV usage for DecisionTreeRegressor model?

I have been implementing a DecisionTreeRegressor model in Anaconda environment with a data set sourced from a 20 million row, 12-dimensional CSV file. I could get the chunks off of the data set with ...
0
votes
0answers
5 views

Ensemble/combining models weighted by number of observations?

Across a few different projects, I have hit a problem where I have two (or more) models: General-Purpose Model: A model which is based on a large amount of data not specifically relevant to my ...
0
votes
0answers
19 views

Stacking Classifier Error

I am using the Stacking classifier and getting following Error CODE ...
3
votes
1answer
46 views

Using output of ML algorithms as an input to different ML algorithm (Ensemble Learning)

I want to assign weights to multiple models and make an single ensemble model. I want to use my outputs as the input to a new machine learning algorithm and the algorithm will learn the correct ...
1
vote
3answers
225 views

Retrieve user features in real time from UserId for prediction

Let's say I'm building an app like Uber and I want to predict the user's most likely destination based on the user's past history, current latitude/longitude, and time/date. Here is the proposed ...
0
votes
1answer
922 views

XGBoost Log Loss different from GridSearchCV Log Loss

I have a classification problem where I am trying to predict if the data returns a 1 or 0. So your classic binary classification. I have my set of data that I have split into the dependent variables (...
0
votes
2answers
89 views

Drawing validation set from test set

I am building a 3 neural network models on dataset that is already separated to train and test sets. From my analysis, I found that this dataset has values on test set which don't exist in the train ...
1
vote
1answer
26 views

When should you use Ensemble?

I have been working on a project as a part of my studies(computer/data science). I tried to make the best classifier I can with what I learned, and recently I have tried to upgrade this classifier ...
1
vote
1answer
245 views

Voting classifier using grid search for Time Series

I have three models: Arima Auto ARIMA Double Exponential Smoothing I would like to apply an ensemble method - a voting method and allow the classifier to learn weights for these three models. I ...
1
vote
1answer
29 views

Making an ensemble model for high F1 score

I presently have 2 algorithms that have a numerical output. Using a threshold of 0.9, I get the classification output. Let's say they are: P (high precision, low recall) R (high recall, low precision)...
0
votes
2answers
41 views

How can compare suggestion models with different performances?

I have 4 class binary classification models. That models identify which class a particular students is suitable for. For example, we have user 1 and 4 classes ...
0
votes
1answer
27 views

Model with 2 datasets: combine time series data and statistics

I am new to data science modelling so apologies if using wrong terminology in advance. I have a standard time series dataset of historical prices which is used to train/test a simple Random Forest ...
0
votes
3answers
215 views

Ensemble Model vs Normal model

If I get 95+ % accuracy in normal models, should I still consider Ensemble models? Why should I choose Ensemble models over normal models?
1
vote
1answer
32 views

How to assign a weight for classifiers when using weighted majority voting?

I am trying to apply weighted majority voting on an ensemble as a combiner method. I read different papers and articles, however, I am still a bit lost on: How the weighted majority voting works How ...
6
votes
1answer
729 views

Is there any difference between a weak learner and a weak classifier?

While reading about decision tree ensembles Gradient Boosting, AdaBoost etc. I have found the following two concepts weak learner and weak classifier. Are they the same? If there is any difference ...
3
votes
2answers
202 views

Probability that ensemble model is correct based on accuracies of its classifiers

I'm trying to understand what I did wrong when trying to answer this question. The exact question is: Assume that we have 3 trained prediction models, and each model outputs either -1 or 1. We then ...
2
votes
2answers
545 views

Is there any way to plot ROC curve for Ensemble hard voting classifier?

I am working on a multi-class text classification problem and performing an Ensemble learning for text classification. I chose hard voting as ensemble technique. I tried to plot ROC curve for my ...
2
votes
1answer
248 views

Why Extra-trees should only be used within ensemble methods?

I was reading scikit-learn documentation for Extremely Randomized Trees and I found this warning: Warning: Extra-trees should only be used within ensemble methods. Why is that?
22
votes
1answer
27k views

Adaboost vs Gradient Boosting

How is AdaBoost different from a Gradient Boosting algorithm since both of them use a Boosting technique? I could not figure out actual difference between these both algorithms from a theory point of ...
7
votes
5answers
2k views

Does ensemble (bagging, boosting, stacking, etc) always at least increase performance?

Ensembling is getting more and more popular. I understand that there are in general three big fields of ensembling, bagging, boosting and stacking. My question is that does the ensembling always at ...
2
votes
3answers
2k views

ValueError: "The estimator should be a classifier"

I am adapting sklearn-extension ELMClassifier to be accepted as base_estimator to both VotingClassifier and AdaboostClassifier. ...
1
vote
2answers
19 views

Selecting models for ensemble from large group of models with high uncertaintly

I'm in a situation where many models have been created, and I have their cross-validation performances as well as performance on test data. I need to select models for inclusion in a simple bagging ...
0
votes
0answers
17 views

How to make an lstm ensemble with different input shapes using keras

This is what I got so far for making an lstm ensemble with one model input for each of the lstm models and for the ensemble model and it works perfectly. ...
1
vote
1answer
28 views

Base model in ensemble learning

I've been doing some research on ensemble learning and read that for base models, model with high variance are often recommended (can't remember which book I read this from exactly). But, it seems ...
4
votes
0answers
2k views
0
votes
0answers
44 views

How do I proceed with model selection?

Only data scientist in an organization and I could really use a sounding board here. In Phase One of a project I deployed four models and served their average as the prediction. I used Random Forest ...
1
vote
0answers
9 views

Large general-purpose model vs ensemble of many smaller models

I am reading this paper - https://arxiv.org/abs/1503.02531v1 - devoted to knowledge distillation in neural networks. One interesting approach is mentioned in this paper in sections ...
1
vote
0answers
41 views

How does stacking help Bias and Variance?

How does stacking help in terms of bias and variance? I have a hunch that stacking can help reduce bias but i am not sure, could someone refer to a paper?
1
vote
0answers
18 views

Different types of Ensemble learning methods [closed]

I've been reading and searching information about different types of Ensemble learning methods however I am a bit confused and want to make sure my understanding is correct. Below is graph of how I ...
0
votes
1answer
53 views

What kind of algorithms can be used as a stacker in stacked generalization?

In stacked generalization, several algorithms (I use some random trees, booster trees, etc.) are first trained and used to make the predictions which are used as input for another algorithm. However, ...
1
vote
1answer
35 views

Augmenting the validation set in Ensemble Model

I have 8 models which I have trained on 90% of my set (training set) and tracked its performance on the loss of the validation set (10% of the original set). I want to generate an ensemble model by ...
0
votes
1answer
66 views

precision recall curve is like stairs [closed]

I am training an ensemble model using a 400 data set sample this led to a precision recall curve that looks like stairs ? what would be the reason beside the low number of samples ?
0
votes
0answers
15 views

Hybrid Ensembling

I read a paper recently where researchers were trying to do classification using ensembling methods. I first read about the concept of BOOSTED-STACKING there.For those who dont know , I can give a ...
0
votes
0answers
17 views

How to interpret gradient descent in boosting ensembles?

I struggle to grasp the role of gradient based optimization in boosting ensembles. As far as I understand boosting means combining a bunch of estimators (of the same types, usually decision trees) ...
2
votes
0answers
78 views

Averaging CNN perform worse than boosting

I'm trying to solve Quora Question Pairs with model stacking. My first layers are: CNN trained to predict the same target as whole model should "Magic features" like question frequency in ...
1
vote
1answer
16 views

How many run should we implement on the machine learning model?

Theoretically, we can implement fix seed on the machine learning model to get the same results every run (reproducible)but it may leads to bias. So, in order to prevent bias, I gonna run the model ...
2
votes
1answer
159 views

How to tune an weighted voting ensemble method?

I am working on kidney cancer patients' data with 5 unbalanced labels. These codes are contained of Normalization, Oversampling on Feature Engineering part. A list of 9 ordinary Machine Learning ...
1
vote
3answers
126 views

Ensembling expressions

I have two models, $m_1$ and $m_2$, and I want to ensemble them into a final model. I want to be able weight one or the other more according to a grid search. There are two main ideas that come to my ...
0
votes
2answers
37 views

What if the votes for 2 classes are equal in an ensemble learning technique?

Suppose in ensemble learning technique, if the number of models that predict class 1 is equal to the number of models that predict class 0. Then, which class will be decided as output?
0
votes
1answer
239 views

How to train with cross validation? and which f1 score to choose?

I got similar results in 2 models which consists of similar algorithms. Model 1 with cv=10 has a f1'micro' of 0.941. See code below. Model 2 only train test split (no cv) has f1'micro' 0.953. Now here ...
2
votes
1answer
61 views

DecisionTreeRegressor under the hood of GradientBoostingClassifier

I'm inspecting the weak estimators of my GradientBoostingClassifier model. This model was fit on a binary class dataset. I noticed that all the weak estimators under this ensemble classifier are ...
3
votes
1answer
306 views

Are "Gradient Boosting Machines (GBM)" and GBDT exactly the same thing?

In the category of Gradient Boosting, I find some terms confusing. I'm aware that XGBoost includes some optimization in comparison to conventional Gradient Boosting. But are Gradient Boosting ...