Questions tagged [ensemble-modeling]

In machine learning, ensemble methods combine multiple algorithms to make a prediction. Bagging, boosting, and stacking, are some examples.

Filter by
Sorted by
Tagged with
0 votes
0 answers
16 views

How to make an ensemble model for classification with pytorch using trained models?

I am trying to make an ensemble model composed of two pre-trained models, using torch, in order to classify an image. Below is some code, based on this post. ...
user avatar
  • 111
1 vote
1 answer
25 views

Random LightGBM Forest

I'm not completly sure about the bias/variance of boosted decision trees (LightGBM especially), thus I wonder if we generally would expect a performance boost by creating an ensemble of multiple ...
user avatar
1 vote
2 answers
26 views

Output of a model as additional input of another model to solve the same task

I was wondering about whether it is possible to train a ML model for a classification task with dataset D, and then train another model to solve the same classification task, which takes as input ...
user avatar
0 votes
0 answers
15 views

Combining features from multiple models and optimising features

I have multiple models predicting an outcome (continuous) and I want to take action to optimize the best values of these features to make a decision. Consider a regression model, y1 = m1x1 + m2x2 + ...
user avatar
0 votes
0 answers
9 views

Is ensemble learning a subset of meta learning?

I'm studying ensemble learning methods, focus on random forest and gradient boost. I read this article about this topic and this about meta learning. It is possible to say that ensemble learning is a ...
user avatar
  • 411
0 votes
1 answer
24 views

Is there a way to combine multiple ML models where each use datasets with different features?

I have a dataset where some features (c,d) apply to only when a feature (a) is a specific value. For example ...
user avatar
  • 1
1 vote
1 answer
18 views

How should I handle time-duration-based columns in classification?

For example, say I am trying to predict whether I will win my next pickleball game. Some features I have are the number of hits, how much water I’ve drinken, etc, and the duration of the match. I’m ...
user avatar
0 votes
1 answer
24 views

Stacking neural nets with cross validation

I am trying to implement stacking model for a ML problem and having hard time figuring out the cross validation strategy. So far I have used 10-fold cross validation for all my models and would like ...
user avatar
0 votes
0 answers
16 views

Combine Content based model and collaborative filtering model

I'm building a ML model for personalization page. I have two models currently one is content based and another is collaborative filtering. Can someone tell me how can I combine both models and use ...
user avatar
  • 163
0 votes
0 answers
15 views

Ensemble Model for Recommendation Engine

I want to build an ensemble recommendation engine where I can combine Surprise library algorithms like KNN and SVD to achieve the best result. Can anyone know how to ensemble this technique?
user avatar
0 votes
0 answers
110 views

SHAP Kernel explainer for ensemble model

I am currently working on a project involving an unsupervised outlier detection ensemble model. However I am getting stuck by an error passed by the shap.KernelExplainer: "The passed model is not ...
user avatar
0 votes
0 answers
8 views

Ensemble of models using different features for different classes

I'm working on a multi-class (4) classification problem - the data set has 88 features and a sample of 1000 data points. I have a hypothesis that there are a different set of features for each class ...
user avatar
0 votes
0 answers
24 views

Constructing heterogeneous datasets

Based on the "Dynamic Ensemble Selection Methods for Heterogeneous Data Mining" paper published by Chris Ballard and Wenjia Wang, I would be grateful if you could guide me on how they ...
user avatar
  • 101
0 votes
0 answers
42 views

How to suppress Python from showing status of my random forest run?

How can I stop scikit-learn Random Forest from displaying the following status output during run? I have already set verbosity to -1 in RandomForestClassifier() and GridSearchCV() ...
user avatar
0 votes
0 answers
221 views

How does XGBoost perform in Parallel

So what I know about boosting technique, Like we train the data and update the weights of falsely predicted values or try to minimize the loss in the next model. So basically, it's the sequential ...
user avatar
  • 163
6 votes
3 answers
499 views

Is it possible to build ensemble models without a decision tree?

Is it possible to build ensemble models without a decision tree? I know that the description of ensembles itself suggests otherwise. However, I am really new to machine learning and all the ensemble ...
user avatar
0 votes
1 answer
16 views

Additional business rules in ensemble methods (RF, Boosted Trees)

How is it possible (if at all) to implement additional business constraints to an ensemble machine learning model, such as random forests or boosted trees? These additional business rules can be ...
user avatar
  • 165
0 votes
0 answers
12 views

How can I use Ensemble learning of two models with different features as an input?

I have a fake news detection problem and it predicts the binary labels "1"&"0" by vectorizing the 'tweet' column, I use three different models for detection but I want to use ...
user avatar
0 votes
0 answers
6 views

Ensemble/combining models weighted by number of observations?

Across a few different projects, I have hit a problem where I have two (or more) models: General-Purpose Model: A model which is based on a large amount of data not specifically relevant to my ...
user avatar
  • 101
0 votes
0 answers
51 views

Stacking Classifier Error

I am using the Stacking classifier and getting following Error CODE ...
user avatar
2 votes
1 answer
67 views

Using output of ML algorithms as an input to different ML algorithm (Ensemble Learning)

I want to assign weights to multiple models and make an single ensemble model. I want to use my outputs as the input to a new machine learning algorithm and the algorithm will learn the correct ...
user avatar
1 vote
1 answer
42 views

Boosting algorithms only built with decision trees? why?

My understanding of boosting is just training models sequentially and learning from its previous mistakes. Can boosting algorithms be built with bunch of logistic regression? or logistic regression + ...
user avatar
  • 343
1 vote
1 answer
33 views

When should you use Ensemble?

I have been working on a project as a part of my studies(computer/data science). I tried to make the best classifier I can with what I learned, and recently I have tried to upgrade this classifier ...
user avatar
0 votes
2 answers
43 views

How can compare suggestion models with different performances?

I have 4 class binary classification models. That models identify which class a particular students is suitable for. For example, we have user 1 and 4 classes ...
user avatar
  • 101
2 votes
1 answer
48 views

Making an ensemble model for high F1 score

I presently have 2 algorithms that have a numerical output. Using a threshold of 0.9, I get the classification output. Let's say they are: P (high precision, low recall) R (high recall, low precision)...
user avatar
1 vote
1 answer
39 views

How to assign a weight for classifiers when using weighted majority voting?

I am trying to apply weighted majority voting on an ensemble as a combiner method. I read different papers and articles, however, I am still a bit lost on: How the weighted majority voting works How ...
user avatar
  • 53
3 votes
2 answers
258 views

Probability that ensemble model is correct based on accuracies of its classifiers

I'm trying to understand what I did wrong when trying to answer this question. The exact question is: Assume that we have 3 trained prediction models, and each model outputs either -1 or 1. We then ...
user avatar
  • 133
3 votes
3 answers
67 views

Understanding Weighted learning in Ensemble Classifiers

I'm currently studying Boosting techniques in Machine Learning and I happened to understand that in Algorithms like Adaboost, each of the training samples is given a weight depending on whether it was ...
user avatar
1 vote
0 answers
13 views

Large general-purpose model vs ensemble of many smaller models

I am reading this paper - https://arxiv.org/abs/1503.02531v1 - devoted to knowledge distillation in neural networks. One interesting approach is mentioned in this paper in sections ...
user avatar
1 vote
1 answer
32 views

Base model in ensemble learning

I've been doing some research on ensemble learning and read that for base models, model with high variance are often recommended (can't remember which book I read this from exactly). But, it seems ...
user avatar
  • 343
1 vote
0 answers
48 views

How does stacking help Bias and Variance?

How does stacking help in terms of bias and variance? I have a hunch that stacking can help reduce bias but i am not sure, could someone refer to a paper?
user avatar
1 vote
0 answers
22 views

Different types of Ensemble learning methods [closed]

I've been reading and searching information about different types of Ensemble learning methods however I am a bit confused and want to make sure my understanding is correct. Below is graph of how I ...
user avatar
  • 343
0 votes
1 answer
107 views

precision recall curve is like stairs [closed]

I am training an ensemble model using a 400 data set sample this led to a precision recall curve that looks like stairs ? what would be the reason beside the low number of samples ?
user avatar
  • 357
0 votes
0 answers
21 views

How to interpret gradient descent in boosting ensembles?

I struggle to grasp the role of gradient based optimization in boosting ensembles. As far as I understand boosting means combining a bunch of estimators (of the same types, usually decision trees) ...
user avatar
  • 857
1 vote
1 answer
21 views

How many run should we implement on the machine learning model?

Theoretically, we can implement fix seed on the machine learning model to get the same results every run (reproducible)but it may leads to bias. So, in order to prevent bias, I gonna run the model ...
user avatar
2 votes
1 answer
248 views

How to tune an weighted voting ensemble method?

I am working on kidney cancer patients' data with 5 unbalanced labels. These codes are contained of Normalization, Oversampling on Feature Engineering part. A list of 9 ordinary Machine Learning ...
user avatar
  • 180
1 vote
3 answers
131 views

Ensembling expressions

I have two models, $m_1$ and $m_2$, and I want to ensemble them into a final model. I want to be able weight one or the other more according to a grid search. There are two main ideas that come to my ...
user avatar
  • 5,584
0 votes
2 answers
65 views

What if the votes for 2 classes are equal in an ensemble learning technique?

Suppose in ensemble learning technique, if the number of models that predict class 1 is equal to the number of models that predict class 0. Then, which class will be decided as output?
user avatar
  • 21
0 votes
1 answer
381 views

How to train with cross validation? and which f1 score to choose?

I got similar results in 2 models which consists of similar algorithms. Model 1 with cv=10 has a f1'micro' of 0.941. See code below. Model 2 only train test split (no cv) has f1'micro' 0.953. Now here ...
user avatar
  • 329
2 votes
1 answer
84 views

DecisionTreeRegressor under the hood of GradientBoostingClassifier

I'm inspecting the weak estimators of my GradientBoostingClassifier model. This model was fit on a binary class dataset. I noticed that all the weak estimators under this ensemble classifier are ...
user avatar
2 votes
1 answer
61 views

How to improve model performace when model shows a systemic pattern in residues

I'm working on a regression model using Boosting algorithms (CatBoost, XGBoost, and LightGBM). All models give similar accuracy of 0.2 RMSE (Target varies from 0 to 1). I obtained the following plots ...
user avatar
  • 161
1 vote
1 answer
318 views

In XGBoost, how is a leaf index corresponding to the particular leaf node in actual base learner trees?

I've trained a XGBoost model for regression, where the max depth is 2. ...
user avatar
3 votes
1 answer
513 views

Are "Gradient Boosting Machines (GBM)" and GBDT exactly the same thing?

In the category of Gradient Boosting, I find some terms confusing. I'm aware that XGBoost includes some optimization in comparison to conventional Gradient Boosting. But are Gradient Boosting ...
user avatar
0 votes
0 answers
25 views

Output all 1 when try to use ensemble CNN on MNIST dataset

...
user avatar
1 vote
0 answers
117 views

Stacking - Appropriate base and meta models

When implementing stacking for model building and prediction (For example using sklearn's StackingRegressor function) what is the appropriate choice of models for the base models and final meta model?...
user avatar
1 vote
1 answer
44 views

How Adaboost calculates error for each weak learner in training?

I am studying the Adaboost classification algorithm because i would like to implement it from scratch. I understand how it works, but i am not able to understand where some steps are placed. I will ...
user avatar
1 vote
1 answer
129 views

How to compare Random Forest with other models

I am new to Machine Learning and I am trying to undrestand the Out of Bag Error in Random Forests and its use. Let's say that we have a dataset. First we use the whole dataset (without splitting it) ...
user avatar
  • 139
1 vote
2 answers
20 views

Selecting models for ensemble from large group of models with high uncertaintly

I'm in a situation where many models have been created, and I have their cross-validation performances as well as performance on test data. I need to select models for inclusion in a simple bagging ...
user avatar
  • 11
1 vote
1 answer
25 views

General practices for building an incremental learning model which never forgets?

I'm new to datascience and appreciate your sage advice! I need to build an incremental learning model, and I know there's a lot that goes into something like that, but I'd like to highlight the most ...
user avatar
  • 151
0 votes
1 answer
29 views

Model with 2 datasets: combine time series data and statistics

I am new to data science modelling so apologies if using wrong terminology in advance. I have a standard time series dataset of historical prices which is used to train/test a simple Random Forest ...
user avatar
  • 1