Questions tagged [ensemble-modeling]

In machine learning, ensemble methods combine multiple algorithms to make a prediction. Bagging, boosting, and stacking, are some examples.

Filter by
Sorted by
Tagged with
0
votes
0answers
19 views

Model selection and hypothesis testing

Only data scientist in an organization and I could really use a sounding board here. In Phase One of a project I deployed four models and served their average as the prediction. I used Random Forest ...
1
vote
1answer
23 views

How to tune an weighted voting ensemble method?

I am working on kidney cancer patients' data with 5 unbalanced labels. These codes are contained of Normalization, Oversampling on Feature Engineering part. A list of 9 ordinary Machine Learning ...
1
vote
3answers
95 views

Ensembling expressions

I have two models, $m_1$ and $m_2$, and I want to ensemble them into a final model. I want to be able weight one or the other more according to a grid search. There are two main ideas that come to my ...
0
votes
2answers
18 views

What if the votes for 2 classes are equal in an ensemble learning technique?

Suppose in ensemble learning technique, if the number of models that predict class 1 is equal to the number of models that predict class 0. Then, which class will be decided as output?
-1
votes
0answers
10 views

Input Output Selection for multiple Models: Ensemble Stacking

I am trying to build an ensemble model. I may use wrong terminology in my question but in essence what my goal is, is to build multiple models who's output goes to a secondary model and I am ...
0
votes
1answer
47 views

How to train with cross validation? and which f1 score to choose?

I got similar results in 2 models which consists of similar algorithms. Model 1 with cv=10 has a f1'micro' of 0.941. See code below. Model 2 only train test split (no cv) has f1'micro' 0.953. Now here ...
2
votes
1answer
26 views

DecisionTreeRegressor under the hood of GradientBoostingClassifier

I'm inspecting the weak estimators of my GradientBoostingClassifier model. This model was fit on a binary class dataset. I noticed that all the weak estimators under this ensemble classifier are ...
0
votes
0answers
10 views

KL Divergence between Predictions and Ground truth

I've got four (non-linear, tree-based) models in production and using the average of them as the served prediction. We get ground truth data immediately. During training the optimized candidate models ...
2
votes
0answers
25 views

How to improve model performace when model shows a systemic pattern in residues

I'm working on a regression model using Boosting algorithms (CatBoost, XGBoost, and LightGBM). All models give similar accuracy of 0.2 RMSE (Target varies from 0 to 1). I obtained the following plots ...
1
vote
1answer
24 views

In XGBoost, how is a leaf index corresponding to the particular leaf node in actual base learner trees?

I've trained a XGBoost model for regression, where the max depth is 2. ...
3
votes
1answer
64 views

Are “Gradient Boosting Machines (GBM)” and GBDT exactly the same thing?

In the category of Gradient Boosting, I find some terms confusing. I'm aware that XGBoost includes some optimization in comparison to conventional Gradient Boosting. But are Gradient Boosting ...
0
votes
0answers
18 views
0
votes
0answers
9 views

A question about SAMME algorithm (Adaboost)

According to this article (which references the original paper), this is the SAMME algorithm for multiclass classification using Adaboost: I would like to understand what is this term in step ...
1
vote
0answers
31 views

Stacking - Appropriate base and meta models

When implementing stacking for model building and prediction (For example using sklearn's StackingRegressor function) what is the appropriate choice of models for the base models and final meta model?...
1
vote
1answer
22 views

How Adaboost calculates error for each weak learner in training?

I am studying the Adaboost classification algorithm because i would like to implement it from scratch. I understand how it works, but i am not able to understand where some steps are placed. I will ...
1
vote
1answer
33 views

How to compare Random Forest with other models

I am new to Machine Learning and I am trying to undrestand the Out of Bag Error in Random Forests and its use. Let's say that we have a dataset. First we use the whole dataset (without splitting it) ...
1
vote
1answer
13 views

Selecting models for ensemble from large group of models with high uncertaintly

I'm in a situation where many models have been created, and I have their cross-validation performances as well as performance on test data. I need to select models for inclusion in a simple bagging ...
1
vote
0answers
8 views

General practices for building an incremental learning model which never forgets?

I'm new to datascience and appreciate your sage advice! I need to build an incremental learning model, and I know there's a lot that goes into something like that, but I'd like to highlight the most ...
0
votes
1answer
23 views

Model with 2 datasets: combine time series data and statistics

I am new to data science modelling so apologies if using wrong terminology in advance. I have a standard time series dataset of historical prices which is used to train/test a simple Random Forest ...
0
votes
0answers
30 views

What is the best way to use Early Stopping in an ensemble (stacking) model?

I have a training and a test dataset. I would like to use the output of Model A in an ensemble model. However, I would like to use early stopping. Usually, I would create Model A for each K-fold (on ...
1
vote
1answer
25 views

Augmenting the validation set in Ensemble Model

I have 8 models which I have trained on 90% of my set (training set) and tracked its performance on the loss of the validation set (10% of the original set). I want to generate an ensemble model by ...
2
votes
1answer
75 views

Is there any way to plot ROC curve for Ensemble hard voting classifier?

I am working on a multi-class text classification problem and performing an Ensemble learning for text classification. I chose hard voting as ensemble technique. I tried to plot ROC curve for my ...
1
vote
0answers
18 views

Decision Boundary with random new observations vs observations from test set

I'm trying to plot decision boundary for Decision Tree classifier. Classifier is trained on training set, and decision boundary (contour) using random new observations and observations from test set ...
4
votes
1answer
38 views

Classification model accuracy with ensemble methods

I came through this statement in a Machine Learning text book based on law of large numbers: Suppose you build an ensemble containing 1,000 classifiers that are individually correct only 51% of the ...
2
votes
1answer
43 views

Stacked Model performance?

I am currently working with a dataset that seems very easily separable and I have an accuracy of 99% for SVM (NN-98%, RF-98%, DT-96-97% and I have checked for leakage & overfitting). As part of my ...
5
votes
2answers
122 views

Stacking and Ensembling methods in Data Science

I understand that using stacking and ensembling has become popular, and these methods can give better results than using a single algorithm. My question is: What are the reasons, statistical or ...
0
votes
1answer
168 views

XGBoost Log Loss different from GridSearchCV Log Loss

I have a classification problem where i am trying to predict if the data returns a 1 or 0. So you're classic binary classification. I have my set of data that I have split into the dependent variables ...
1
vote
1answer
30 views

How does bagging help reduce the variance

I learned that bagging helps reduce variance by averaging but I couldn't understand this. Can someone explain this intuitively?
0
votes
1answer
159 views

How do I Combine two CNN models (h5 format)?

I have two CNN models, both of them are trained on the same dataset. How do I combine/ensemble both to make predictions on test data? ...
1
vote
1answer
238 views

What is the difference between ensemble methods and hybrid methods, or is there none?

I have the feeling that these terms often are used as synonyms for one another, however they have the same goal, namely increasing prediction accuracy by combining different algorithms. My question ...
0
votes
0answers
9 views

Ensemble Model to Handle Different Image Attributes

I'm working on a project where I have images annotated across several attributes, say X, Y, Z...
1
vote
1answer
21 views

Model stacking with instance attributes [duplicate]

From what I have understood about model stacking: the meta estimator trains to combine the N-models predictions to fit the ground truth. Once trained, it combines the 1st level output to approach the ...
1
vote
1answer
20 views

What is the form of data used for prediction with generalized stacking ensemble?

I am very confused as to how training data is split and on what data level 0 predictions are made when using generalized stacking. This question is similar to mine, but the answer is not sufficiently ...
0
votes
1answer
15 views

Low leves of probability observed after modelling.Is it right to scale the probability

I have done modelling on imbalanced class , without any sampling methods. Event rate is around 0.1 ,After modelling I am getting probalities more at the lower side close to zero.I have tried differnt ...
1
vote
1answer
23 views

Can I use SVC() as a base_estimtor for ensemble methods?

I am currently testing out a few different ensemble methods on my dataset. I've heard that you can also use support vector machines as base learners in boosting and bagging methods but I am not sure ...
1
vote
0answers
37 views

Can bagging ensemble consist of heterogeneous base models?

Bagging or bootstrap aggregation seems to make sense for time series forecasting using an ensemble because bagging randomizes subsets of the data with replacement. However, I've only seen bagging used ...
0
votes
0answers
14 views

Creating a sub-model from pre-trained model

I have a pre-trained model having the following architecture: ...
0
votes
0answers
7 views

Domain specific language to describe ensemble model

I'm looking for some tool/library/widely used approach to describe Hierarchical Model structure like ensemble: It's absolutely straightforward how to do it with simple ensemble like It can be ...
0
votes
1answer
91 views

Can you combine two xgboost models into one?

If you have built two different xgbost models, with say 100 trees each, is it possible to combine into an xgboost model with 200 trees?
0
votes
1answer
189 views

What is the selection criteria to choose between XGBoost and Random Forest

I am trying to understand - when would someone choose Random Forest over XGBoost and vice versa. All the articles out there highlights on the differences between both. I understand them. But when ...
2
votes
2answers
625 views

ValueError: “The estimator should be a classifier”

I am adapting sklearn-extension ELMClassifier to be accepted as base_estimator to both VotingClassifier and AdaboostClassifier. ...
0
votes
0answers
16 views

Stacking using layer-1 models predictions on test set

I am new to Data Science and have been studying the methods of stacking to find out if it can meet the following fact, but I did not find or understand evidence that it can or cannot work. Let's ...
0
votes
0answers
23 views

Model ensemble - using model associated with median (instead of mean) for calculation purposes (Explainability)

I have seen many model ensemble litterature. Most, if not all of it, consider averaging models. I was considering using the median instead of the mean. In general I would consider this a good ...
1
vote
1answer
24 views

What should I use as training data for base (level 1) classifiers in ensembling?

Can I just take all training data that I have, train the base models on them and then take their results and use them for training level 2 model? Is this a good practice, or should it be done ...
1
vote
1answer
38 views

How many models shall be used in ensemble modelling?

I wish to make an ensemble of deep models to solve a classification problem. I want to know how many models shall be used to create that ensemble to ensure unbiased results. I have head 30+ models ...
0
votes
1answer
139 views

What's wrong with RF/SVM with word embedding (GloVe)?

I searched many times in google for examples on word embedding (specifically GloVe) with Random forest and I couldn't find any single example. For GloVe, it was all either LSTM or CNN. Maybe there's ...
2
votes
1answer
18 views

is it possible to use the train result as another feature and retrain?

is it possible to use the train result as another feature and retrain? for example I make prediction with classification and add this result to the table and train xgboost?
0
votes
0answers
23 views

Ensemble two models

I have regression task and I am predicting here with linear regression and randomforest models. Need some hints or code example how to ensemble them (averaging already done). Here are my model ...
2
votes
2answers
94 views

Hyperparameter optimization, ensembling instead of selecting with CV criteria

While burning CPUs performing a CV selection on a thin grid put on some hyperparameter space. I am using the `scikit-learn' API, for which the end result is a single point on the hyperparameter space, ...
1
vote
0answers
12 views

How to Predict Employee count of businesses using Keras classifiers

I am trying to predict the amount of employees a business has based on a set of input variables. I am using things like the business's age, transaction details, geographic location, business structure ...