Questions tagged [ensemble-modeling]

In machine learning, ensemble methods combine multiple algorithms to make a prediction. Bagging, boosting, and stacking, are some examples.

Filter by
Sorted by
Tagged with
0
votes
1answer
17 views

Can you combine two xgboost models into one?

If you have built two different xgbost models, with say 100 trees each, is it possible to combine into an xgboost model with 200 trees?
0
votes
1answer
27 views

What is the selection criteria to choose between XGBoost and Random Forest

I am trying to understand - when would someone choose Random Forest over XGBoost and vice versa. All the articles out there highlights on the differences between both. I understand them. But when ...
0
votes
0answers
28 views

ValueError: “The estimator should be a classifier”

I am adapting sklearn-extension ELMClassifier to be accepted as base_estimator to both VotingClassifier and AdaboostClassifier. ...
0
votes
0answers
13 views

Stacking using layer-1 models predictions on test set

I am new to Data Science and have been studying the methods of stacking to find out if it can meet the following fact, but I did not find or understand evidence that it can or cannot work. Let's ...
0
votes
0answers
10 views

Model ensemble - using model associated with median (instead of mean) for calculation purposes (Explainability)

I have seen many model ensemble litterature. Most, if not all of it, consider averaging models. I was considering using the median instead of the mean. In general I would consider this a good ...
1
vote
1answer
23 views

What should I use as training data for base (level 1) classifiers in ensembling?

Can I just take all training data that I have, train the base models on them and then take their results and use them for training level 2 model? Is this a good practice, or should it be done ...
1
vote
1answer
29 views

How many models shall be used in ensemble modelling?

I wish to make an ensemble of deep models to solve a classification problem. I want to know how many models shall be used to create that ensemble to ensure unbiased results. I have head 30+ models ...
0
votes
1answer
54 views

What's wrong with RF/SVM with word embedding (GloVe)?

I searched many times in google for examples on word embedding (specifically GloVe) with Random forest and I couldn't find any single example. For GloVe, it was all either LSTM or CNN. Maybe there's ...
2
votes
1answer
14 views

is it possible to use the train result as another feature and retrain?

is it possible to use the train result as another feature and retrain? for example I make prediction with classification and add this result to the table and train xgboost?
0
votes
0answers
19 views

Ensemble two models

I have regression task and I am predicting here with linear regression and randomforest models. Need some hints or code example how to ensemble them (averaging already done). Here are my model ...
2
votes
2answers
89 views

Hyperparameter optimization, ensembling instead of selecting with CV criteria

While burning CPUs performing a CV selection on a thin grid put on some hyperparameter space. I am using the `scikit-learn' API, for which the end result is a single point on the hyperparameter space, ...
1
vote
0answers
12 views

How to Predict Employee count of businesses using Keras classifiers

I am trying to predict the amount of employees a business has based on a set of input variables. I am using things like the business's age, transaction details, geographic location, business structure ...
2
votes
0answers
15 views

How to estimate the marginal distribution of a class with respect to one predictor in a classification task?

I have a dataset with a binary dependent variable $y \in \{0,1\}$ and a set of predictors $x1,x2,..,t$. Here, $t$ is the time in minutes (in 24 hrs, that is $t \in (0,1440)$). I want to estimate the ...
1
vote
1answer
14 views

When should we start using stacking of models?

I am solving a Kaggle contest and my single model has reached score of 0.121, I'd like to know when to start using ensembling/stacking to improve the score. I used lasso and xgboost and there ...
4
votes
2answers
181 views

SHAP value can explain right?

I face a problem with using SHAP value to interpret the Tree-based model. (https://github.com/slundberg/shapsd) First, I have input around 30 features and I have 2 features that have high positive ...
0
votes
0answers
15 views

Combining several Multi-Output-Models into a single Multi-Output-Model

I'm trying to create a k-Nearest-Neighbor based model of 76-dimensional input data $I$ and 44-dimensional output data $O$. Through domain knowledge I know that only certain input dimensions are ...
1
vote
0answers
15 views

Is there a universal convergence rate when stacking models/experts?

It's fairly common to see people stacking different models when chasing marginal gains in contexts such as Kaggle competitions or the Netflix challenge. I would like to know about the mathematics ...
0
votes
1answer
36 views

Deep model ensemble giving different results

I am making an ensemble of deep models for solving a classification problem. The initial weights follow the default distribution of keras layers. Each time I run ...
2
votes
2answers
74 views

Random forest vs majority voting

I'm using spark with scala to implement majority voting of decision trees and random forest (both are configured in the same way - same depth, the same amount of base classifiers etc.). Dataset is ...
1
vote
1answer
76 views

Combining multiple neural networks with different activation functions

I have 3 neural networks where each has as a different activation function: Sigmoid, Tanh and Softmax. I am planning to average their final predictions, but as we know the functions doesn't have the ...
0
votes
2answers
32 views

Drawing validation set from test set

I am building a 3 neural network models on dataset that is already separated to train and test sets. From my analysis, I found that this dataset has values on test set which don't exist in the train ...
1
vote
2answers
28 views

Ensemble Techniques - Bagging | Subset size

I do have a question on ensemble techniques Baggging/Boosting. - What would be the subset size for Bagging?
0
votes
1answer
21 views

Ensemble Techniques - Boosting

I understand boosting is a sequential learning technique and it use the prediction from previous model as a dataset for new model ,after adding weight to the misclassified data points. The point ...
1
vote
1answer
521 views

Training an ensemble of small neural networks efficiently in TensorFlow 2

I have a bunch of small neural networks (say, 5 to 50 feed-forward neural networks with only two hidden layers with 10-100 neurons each), which differ only in the weight initialization. I want to ...
2
votes
2answers
80 views

Gradient boosting how can accuracy increase when we lower the depth of tree?

What I don't understand about gradient boost is, doesn't lowering height of the tree means we use fewer features in our model? From my model I get the highest accuracy when the depth is one. Meaning ...
1
vote
0answers
9 views

Named Entity Recognition for Sesotho sa Leboa (Sepedi) one of the South African Official Language

I'm looking for model to used to develop NER for Sepedi?
0
votes
1answer
48 views

How to optimize hyperparameters in stacked model?

I was wondering whether somebody could explain how to optimize hyperparameters for the base learners and meta algorithm when stacking? In many tutorials they seem to be plucked out of thin air! ...
0
votes
0answers
29 views

How to add weight to a variable to a ensemble method

I am building an ensemble method and would like to add weight to one particular variable (x3 in this case). How can I do this? Here is my code: ...
2
votes
1answer
61 views

How are Decision Trees averaged in Random Forest?

We all know that a Random Forest is an ensemble of Decision Trees, whose results are averaged. Every source I find simply talks about "averaging trees", but how does this "averaging of trees" happens?...
0
votes
0answers
15 views

Manual setting of target variable based on features' minimum values: f1 score = 1

I am building a classifier for user engagement in my website. Basically, since there are no "proxy" for engagement, i.e. there is no pre-defined target variable, I came up with minimum thresholds ...
3
votes
1answer
65 views

Weighted Linear Combination of Classifiers

I am trying to build an ensemble of classifiers whereby I want my algorithm to learn a set of weights such that it can weight the outputs of different classifiers for a set of data points. I am ...
0
votes
0answers
34 views

How to deal with small amount of labeled samples?

I'm trying to develop skill to deal with very small amount of labeled samples (250 labeled/20000 total, 200 features) practicing on Kaggle "Don't Overfit" dataset (Traget_Practice have provided all 20,...
1
vote
0answers
12 views

Ensemble learning for multiple hypothesis classes

Just to confirm if the following description falls in the category of ensemble learning. Suppose given a training set $D=\{(X,Y)\}$ we are asked to train a regressor. But now the way we do it is to ...
1
vote
1answer
47 views

Understanding why ensembling only improves marginally

I have two models, A and B, trained on Imagenet. Their accuracies on Imagenet validation set are 35.6% and 28.64% respectively, while the accuracy of their ensemble (averaging their scores) is 35.68%. ...
1
vote
0answers
12 views

Would it make technical sense to model its two known subelements separately (CLV example)?

Let's assume Customer Lifetime Value (CLV) is defined as average basket x frequency. Option A is to build model predicting CLV ...
1
vote
1answer
202 views

checking model stability - Performance for different class

I tried to do multi-class classification problem. The goal is to predict whether the match will be won by HomeTeam, AwayTeam or Draw. I did feature engineering from the attributes and finally came up ...
1
vote
0answers
68 views

Recommendations for Neural Network Stacking Project

I embarked in a project to use a stacking ensemble of neural networks to perform a binary classification prediction. I have the ensemble running and making decent accuracy estimates (90% ca.). ...
2
votes
1answer
2k views

Is there any implementation of Extended Isolation Forest algorithm in R/Python?

I am using isofor package for regular Isolation Forest but I came by an article about Extended Isolation Forest and need your advise which package has this ...
1
vote
0answers
175 views

Ensemble Method using XGBoost and RotationForest python

How can I create an ensemble model using XGBoost and Rotation Forest in Python?
1
vote
1answer
140 views

Voting classifier using grid search for Time Series

I have three models: Arima Auto ARIMA Double Exponential Smoothing I would like to apply an ensemble method - a voting method and allow the classifier to learn weights for these three models. I ...
3
votes
1answer
42 views

Building Stacking machine learning model using three base classifiers

I did a stacking using three base classifiers RF, NB, KN N and metamodel random forest or SVM using sklearn library But which is strange each time i change the metamodel i got the same results. Is it ...
3
votes
0answers
44 views

Is linear regression on the trees of XGBoost (rather than taking their mean) useful/popular?

Given training data $(\underline{x}_1, y_1),...,(\underline{x_N}, y_N)$, one can choose a variety of ensemble method for trees. These algorithms output a set of trees $T_1, ..., T_n$, and then the ...
1
vote
1answer
574 views

Difference between bagging and boosting

Can anyone explain me the basic difference between bagging and boosting and which technique can be used in which scenario?
9
votes
1answer
12k views

Adaboost vs Gradient Boosting

How AdaBoost is different than Gradient Boosting algorithm since both of them works on Boosting technique? I could not figure out actual difference between these both algorithms from theory point of ...
1
vote
1answer
247 views

best activation function for ensemble?

i have created some logistic regression model (different preprocessing) with softmax function. and i mix all model with an ensemble with a hierarchical method. so the output of all model (base) will ...
2
votes
2answers
51 views

What methods can be used to detect duplicacy in image dataset?

I want to remove duplicate images from a dataset of 50Million images. What is the best method to detect all the duplicates? Do you think one shot learning is good for this?
1
vote
1answer
194 views

Why Extra-trees should only be used within ensemble methods?

I was reading scikit-learn documentation for Extremely Randomized Trees and I found this warning: Warning: Extra-trees should only be used within ensemble methods. Why is that?
1
vote
1answer
65 views

Combining outputs of ridge regression models?

I am facing an issue where I have 7 sets of different variables/columns/predictors. I am trying to predict same target variable and I want to observe the importance/effect of all the sets according ...
1
vote
0answers
384 views

Combining image and scalar inputs into a neural network

I'm looking at the best way of combining CNN with image input and a scalar value. I know that one of the ways is to concatenate flatten layer with this scalar value. But flatten layer consist for ...
1
vote
0answers
72 views

heterogeneous input features for estimators in sklearn.ensemble.VotingClassifier

I would like to do ensemble on my model. Two of them are SVM and XGBoosting. SVM could not tolerate null value and XGB can do it. So I have different features for each of them. but when ...