Questions tagged [ensemble-modeling]

In machine learning, ensemble methods combine multiple algorithms to make a prediction. Bagging, boosting, and stacking, are some examples.

Filter by
Sorted by
Tagged with
0 votes
1 answer
13 views

Training XGBoost on time series features of varying sample length

I have some time series data that contain features that that go back anywhere from 5 to 50 years. I've considered imputation (e.g. taking the mean), but I'm not sure it's feasible to impute such large ...
0 votes
0 answers
6 views

Combining DeepXDE (physics-informed neural networks) with other tensorflow models

I would like to stack two models from scikit-learn and tensorflow. I have tried to make an illustration of what I want to do. What I am looking for is the actual wrapper. Does scikit-learn have any ...
0 votes
1 answer
18 views

Where should I stop training if I want to bag models

Let's say I have a clear case of overfitting where my loss curves look like this (x axis are iterations): Now I would like to try bagging to reduce the variance, where should I stop models training? ...
  • 101
0 votes
1 answer
51 views

Found input variables with inconsistent numbers of samples: ValueError

Today I am trying build ensemble model. Where I am working with iris dataset. In my model I am using ...
0 votes
1 answer
22 views

Understanding Classification model results

In a certain binary classifcation problem I am getting a AUC of 1 and Accuracy,FI,Recall,Precision of ~99.7 both in train,test and holdout sets. But when I run the model on unlabelled data which I ...
  • 101
0 votes
0 answers
30 views

What is a good way to model this?

I have a dataframe df which looks like this : user Date TP_A TP_C TP_D TP_E TP_B TP_F Order 1 11-07-2014 0 0 1 0 0 0 0 1 11-07-2014 0 0 0 1 0 0 0 1 15-07-2014 0 ...
  • 101
0 votes
0 answers
16 views

Why would an ensemble model perform worse than all individual models? biomod2

I'm using biomod2 in R and my ensemble model performs worse on the evaluation data (drastically lower ROC, 0.835) than any of the individual models (ROC ranges 0.89-0.97). What could be causing this? ...
0 votes
2 answers
34 views

If you are making a ensemble model does training data on base models have to be different from one another

I was reading this article talking about ensemble models. I was interested in the max voting model using 3 base learners. However, I am a little confused about the process. Currently, I'm thinking it ...
  • 13
0 votes
0 answers
9 views

Univariate Time Series Revenue Forecast Flow for Multiple Products (Different Products in Same Domain)

My task is revenue forecast. I would like predict 7 days horizon for each 10 products. I am planning to use ensemble model. Can 10 products(same domain, for ex: ios game app revenues) be predicted ...
  • 101
2 votes
2 answers
143 views

Are feature importances of ensemble methods sensible interpretable?

As mentioned in the question, it is easy to interpret the meaning of features in algorithms like simple decision trees. But in the case of ensemble methods that are known to average/modify features, ...
  • 23
0 votes
1 answer
24 views

Voting Regression models, other approaches than averaging the results from each estimators

In a regression problem that I'm currently working on, it seems that my model is doing well on higher values but significantly worse on lower values (e.g. values from 100,000,000 to 105,000,000 are ...
  • 103
0 votes
0 answers
12 views

Can I use different dataset when performing model stacking?

Let's say I want to detect new species of fish. I have several models, each trained to recognize a different characteristic, e.g., the speed of known fish, the size of known fish, their known shapes, ...
  • 121
1 vote
1 answer
15 views

Outputs from models (trained on different data) as inputs to another model?

Let's say I want to detect new species of fish. I have several models, each trained to recognize a different characteristic, e.g., the speed of known fish, the size of known fish, their known shapes, ...
  • 121
1 vote
1 answer
73 views

What is meant by this notation for ensemble classifier error rate

The below is a picture which denotes the error of an ensemble classifier. Can someone help me understand the notation What does it mean to have (25 and i) in brackets and what is ε^1 is it error of ...
  • 125
0 votes
1 answer
338 views

How to make an ensemble model for classification with pytorch using trained models?

I am trying to make an ensemble model composed of two pre-trained models, using torch, in order to classify an image. Below is some code, based on this post. ...
  • 121
1 vote
1 answer
98 views

Random LightGBM Forest

I'm not completly sure about the bias/variance of boosted decision trees (LightGBM especially), thus I wonder if we generally would expect a performance boost by creating an ensemble of multiple ...
1 vote
2 answers
672 views

Output of a model as additional input of another model to solve the same task

I was wondering about whether it is possible to train a ML model for a classification task with dataset D, and then train another model to solve the same classification task, which takes as input ...
0 votes
0 answers
15 views

Combining features from multiple models and optimising features

I have multiple models predicting an outcome (continuous) and I want to take action to optimize the best values of these features to make a decision. Consider a regression model, y1 = m1x1 + m2x2 + ...
1 vote
1 answer
25 views

Is ensemble learning a subset of meta learning?

I'm studying ensemble learning methods, focusing on random forest and gradient boost. I read this article about this topic and this about meta-learning. It is possible to say that ensemble learning is ...
  • 471
0 votes
1 answer
235 views

Is there a way to combine multiple ML models where each use datasets with different features?

I have a dataset where some features (c,d) apply to only when a feature (a) is a specific value. For example ...
  • 1
1 vote
1 answer
20 views

How should I handle time-duration-based columns in classification?

For example, say I am trying to predict whether I will win my next pickleball game. Some features I have are the number of hits, how much water I’ve drinken, etc, and the duration of the match. I’m ...
0 votes
1 answer
32 views

Stacking neural nets with cross validation

I am trying to implement stacking model for a ML problem and having hard time figuring out the cross validation strategy. So far I have used 10-fold cross validation for all my models and would like ...
0 votes
0 answers
22 views

Combine Content based model and collaborative filtering model

I'm building a ML model for personalization page. I have two models currently one is content based and another is collaborative filtering. Can someone tell me how can I combine both models and use ...
  • 183
0 votes
0 answers
22 views

Ensemble Model for Recommendation Engine

I want to build an ensemble recommendation engine where I can combine Surprise library algorithms like KNN and SVD to achieve the best result. Can anyone know how to ensemble this technique?
0 votes
0 answers
200 views

SHAP Kernel explainer for ensemble model

I am currently working on a project involving an unsupervised outlier detection ensemble model. However I am getting stuck by an error passed by the shap.KernelExplainer: "The passed model is not ...
0 votes
0 answers
10 views

Ensemble of models using different features for different classes

I'm working on a multi-class (4) classification problem - the data set has 88 features and a sample of 1000 data points. I have a hypothesis that there are a different set of features for each class ...
0 votes
0 answers
25 views

Constructing heterogeneous datasets

Based on the "Dynamic Ensemble Selection Methods for Heterogeneous Data Mining" paper published by Chris Ballard and Wenjia Wang, I would be grateful if you could guide me on how they ...
  • 101
0 votes
0 answers
74 views

How to suppress Python from showing status of my random forest run?

How can I stop scikit-learn Random Forest from displaying the following status output during run? I have already set verbosity to -1 in RandomForestClassifier() and GridSearchCV() ...
0 votes
1 answer
497 views

How does XGBoost perform in Parallel

So what I know about boosting technique, Like we train the data and update the weights of falsely predicted values or try to minimize the loss in the next model. So basically, it's the sequential ...
  • 183
6 votes
3 answers
593 views

Is it possible to build ensemble models without a decision tree?

Is it possible to build ensemble models without a decision tree? I know that the description of ensembles itself suggests otherwise. However, I am really new to machine learning and all the ensemble ...
0 votes
1 answer
17 views

Additional business rules in ensemble methods (RF, Boosted Trees)

How is it possible (if at all) to implement additional business constraints to an ensemble machine learning model, such as random forests or boosted trees? These additional business rules can be ...
  • 165
2 votes
1 answer
96 views

Using output of ML algorithms as an input to different ML algorithm (Ensemble Learning)

I want to assign weights to multiple models and make an single ensemble model. I want to use my outputs as the input to a new machine learning algorithm and the algorithm will learn the correct ...
1 vote
1 answer
43 views

Boosting algorithms only built with decision trees? why?

My understanding of boosting is just training models sequentially and learning from its previous mistakes. Can boosting algorithms be built with bunch of logistic regression? or logistic regression + ...
  • 385
2 votes
1 answer
41 views

When should you use Ensemble?

I have been working on a project as a part of my studies(computer/data science). I tried to make the best classifier I can with what I learned, and recently I have tried to upgrade this classifier ...
0 votes
2 answers
44 views

How can compare suggestion models with different performances?

I have 4 class binary classification models. That models identify which class a particular students is suitable for. For example, we have user 1 and 4 classes ...
  • 101
2 votes
1 answer
70 views

Making an ensemble model for high F1 score

I presently have 2 algorithms that have a numerical output. Using a threshold of 0.9, I get the classification output. Let's say they are: P (high precision, low recall) R (high recall, low precision)...
1 vote
1 answer
51 views

How to assign a weight for classifiers when using weighted majority voting?

I am trying to apply weighted majority voting on an ensemble as a combiner method. I read different papers and articles, however, I am still a bit lost on: How the weighted majority voting works How ...
  • 53
3 votes
2 answers
304 views

Probability that ensemble model is correct based on accuracies of its classifiers

I'm trying to understand what I did wrong when trying to answer this question. The exact question is: Assume that we have 3 trained prediction models, and each model outputs either -1 or 1. We then ...
  • 133
3 votes
3 answers
70 views

Understanding Weighted learning in Ensemble Classifiers

I'm currently studying Boosting techniques in Machine Learning and I happened to understand that in Algorithms like Adaboost, each of the training samples is given a weight depending on whether it was ...
1 vote
0 answers
19 views

Large general-purpose model vs ensemble of many smaller models

I am reading this paper - https://arxiv.org/abs/1503.02531v1 - devoted to knowledge distillation in neural networks. One interesting approach is mentioned in this paper in sections ...
1 vote
1 answer
33 views

Base model in ensemble learning

I've been doing some research on ensemble learning and read that for base models, model with high variance are often recommended (can't remember which book I read this from exactly). But, it seems ...
  • 385
1 vote
0 answers
79 views

How does stacking help Bias and Variance?

How does stacking help in terms of bias and variance? I have a hunch that stacking can help reduce bias but i am not sure, could someone refer to a paper?
1 vote
0 answers
24 views

Different types of Ensemble learning methods [closed]

I've been reading and searching information about different types of Ensemble learning methods however I am a bit confused and want to make sure my understanding is correct. Below is graph of how I ...
  • 385
0 votes
1 answer
118 views

precision recall curve is like stairs [closed]

I am training an ensemble model using a 400 data set sample this led to a precision recall curve that looks like stairs ? what would be the reason beside the low number of samples ?
  • 367
0 votes
0 answers
25 views

How to interpret gradient descent in boosting ensembles?

I struggle to grasp the role of gradient based optimization in boosting ensembles. As far as I understand boosting means combining a bunch of estimators (of the same types, usually decision trees) ...
  • 947
1 vote
1 answer
27 views

How many run should we implement on the machine learning model?

Theoretically, we can implement fix seed on the machine learning model to get the same results every run (reproducible)but it may leads to bias. So, in order to prevent bias, I gonna run the model ...
2 votes
1 answer
315 views

How to tune an weighted voting ensemble method?

I am working on kidney cancer patients' data with 5 unbalanced labels. These codes are contained of Normalization, Oversampling on Feature Engineering part. A list of 9 ordinary Machine Learning ...
  • 180
1 vote
3 answers
135 views

Ensembling expressions

I have two models, $m_1$ and $m_2$, and I want to ensemble them into a final model. I want to be able weight one or the other more according to a grid search. There are two main ideas that come to my ...
  • 5,714
0 votes
2 answers
132 views

What if the votes for 2 classes are equal in an ensemble learning technique?

Suppose in ensemble learning technique, if the number of models that predict class 1 is equal to the number of models that predict class 0. Then, which class will be decided as output?
  • 21
0 votes
1 answer
421 views

How to train with cross validation? and which f1 score to choose?

I got similar results in 2 models which consists of similar algorithms. Model 1 with cv=10 has a f1'micro' of 0.941. See code below. Model 2 only train test split (no cv) has f1'micro' 0.953. Now here ...
  • 329