Questions tagged [ensemble-learning]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
0answers
32 views

Can numerical encoding really replace one-hot encoding?

I am reading these articles (see below), which advocate the use of numerical encoding rather than one hot encoding for better interpretability of feature importance output from ensemble models. This ...
0
votes
1answer
64 views

Feature Selection using Stacking Ensemble?

I want to combine some estimators, such as Logistic Regression, Gaussian NB and ...
1
vote
1answer
35 views

How to assign a weight for classifiers when using weighted majority voting?

I am trying to apply weighted majority voting on an ensemble as a combiner method. I read different papers and articles, however, I am still a bit lost on: How the weighted majority voting works How ...
0
votes
0answers
7 views

How to apply feature selection in cross validated bagging

Normally in cross validation decision tree, feature selection will occur with training data but in bagging ensemble the training data is bootstrapped. How can I apply feature selection in cross ...
1
vote
0answers
20 views

Ensemble of different reservoirs (echo state networks)

Suppose I want to do reservoir computing to classify the input to the proper category (e.g. recognizing a handwritten letter). Ideally, after training a single reservoir and testing it, there would be ...
0
votes
0answers
24 views

Any insights on gradient boosting algorithms for classification task with large class number?

Recently I have been learning Gradient Boosting (GB) on multi-class classification. I searched Google Scholar for a while and found one paper on multi-class classification using GB 1. However, little ...
0
votes
1answer
58 views

Is there a closed formula/function for decision trees?

i've been studying gradient boosting so realize the pure algorithm requires a function F/model to get boosted.What is the explicit F on gradient boosting trees?
0
votes
2answers
46 views

What if the votes for 2 classes are equal in an ensemble learning technique?

Suppose in ensemble learning technique, if the number of models that predict class 1 is equal to the number of models that predict class 0. Then, which class will be decided as output?
0
votes
1answer
334 views

How to train with cross validation? and which f1 score to choose?

I got similar results in 2 models which consists of similar algorithms. Model 1 with cv=10 has a f1'micro' of 0.941. See code below. Model 2 only train test split (no cv) has f1'micro' 0.953. Now here ...
1
vote
1answer
250 views

In XGBoost, how is a leaf index corresponding to the particular leaf node in actual base learner trees?

I've trained a XGBoost model for regression, where the max depth is 2. ...
3
votes
1answer
394 views

Are "Gradient Boosting Machines (GBM)" and GBDT exactly the same thing?

In the category of Gradient Boosting, I find some terms confusing. I'm aware that XGBoost includes some optimization in comparison to conventional Gradient Boosting. But are Gradient Boosting ...
1
vote
0answers
108 views

Stacking - Appropriate base and meta models

When implementing stacking for model building and prediction (For example using sklearn's StackingRegressor function) what is the appropriate choice of models for the base models and final meta model?...
0
votes
1answer
53 views

Incorrect multi-variate anomaly detection - Isolation Forest Python

My data looks like below. it has 333 rows and 2 columns. Clearly the first row is anomaly. ndf: ...
1
vote
1answer
2k views

What is the difference between ensemble methods and hybrid methods, or is there none?

I have the feeling that these terms often are used as synonyms for one another, however they have the same goal, namely increasing prediction accuracy by combining different algorithms. My question ...
1
vote
1answer
33 views

What is the form of data used for prediction with generalized stacking ensemble?

I am very confused as to how training data is split and on what data level 0 predictions are made when using generalized stacking. This question is similar to mine, but the answer is not sufficiently ...
1
vote
1answer
127 views

How can I improve my model on a very very small dataset?

I am starting as a PhD student and we want to find appropriate materials (with certain qualities) from basic chemical properties like charge, etc. There are a lot of models and datasets in similar ...
1
vote
0answers
3k views

ValueError: Graph disconnected: cannot obtain value for tensor Tensor

I'm trying to perform a stacking ensemble of three VGG-16 models, all custom-trained on my personal dataset and having the same input shape. This is the code: ...
2
votes
1answer
35 views

Neural Network Multiple | Averging predictions

I am training multiple neural networks with various parameters. I am trying to average their predictions, but I am not really sure what that means, I am confused about what to average exactly. Here is ...
1
vote
2answers
126 views

Ensemble Techniques - Bagging | Subset size

I do have a question on ensemble techniques Baggging/Boosting. - What would be the subset size for Bagging?
0
votes
1answer
36 views

Ensemble Techniques - Boosting

I understand boosting is a sequential learning technique and it use the prediction from previous model as a dataset for new model ,after adding weight to the misclassified data points. The point ...
1
vote
1answer
216 views

Bagging with Neural Networks Best practices

I am trying to build a majority vote system for 3 Neural Networks, and I came across the concept of Bagging method. Actually, I want to use neural networks as weak learners (I know it's debatable, but ...
1
vote
0answers
37 views

How to choose learning models for different levels of a Stacking Ensemble?

I was wondering if there is any specific rule for designing the architecture of a Stacking Ensemble (e.g. number of levels, models in each level, and models in general). Also, can there be an overlap ...
-1
votes
1answer
36 views

If There is a case where decision trees are getting overfitted so by using gradient boost method do we solve that problem?

I have came across a case where my decision trees are getting overfitting so by using methods like gradient boost can I solve that problem.
1
vote
1answer
139 views

Can we use boosting algorithms like Adaboost and gradient boosting with only one classifier

I have been working on ensemble learning and I came across this doubt that unlike other ensemble learning algorithms like voting classifier a can we only use one classifier with boosting.
1
vote
0answers
40 views

Ensemble technique for combining predictions from classification and regression algorithms

Given an anomaly detection problem - A, I have divided the problem into two independent subtasks -A1 and ...
1
vote
0answers
14 views

Ensemble learning for multiple hypothesis classes

Just to confirm if the following description falls in the category of ensemble learning. Suppose given a training set $D=\{(X,Y)\}$ we are asked to train a regressor. But now the way we do it is to ...
4
votes
2answers
1k views

How to apply Stacking cross validation for time-series data?

Normally stacking algorithm uses K-fold cross validation technique to predict oof validation that used for level 2 prediction. In case of time-series data (say stock movement prediction), K-fold ...
1
vote
1answer
1k views

Difference between bagging and boosting

Can anyone explain me the basic difference between bagging and boosting and which technique can be used in which scenario?
2
votes
1answer
253 views

Why Extra-trees should only be used within ensemble methods?

I was reading scikit-learn documentation for Extremely Randomized Trees and I found this warning: Warning: Extra-trees should only be used within ensemble methods. Why is that?
0
votes
1answer
67 views

Collection of several learners [closed]

I have few questions for which I could not extract answers from text books and online tutorials. Therefore, will be extremely grateful if the following points are clarified. 1) If I want to apply SVM,...
3
votes
0answers
70 views

Stacking when the the target variable is categorical?

I'm trying to use stacking when predicting for the infamous Iris dataset. Also, I'd like to build to stacked classifier by myself which means I don't want to use mlxtend because it's too "easy" and ...
1
vote
1answer
76 views

How to construct a self learning process for an ensemble model?

I have two different (regression) models spitting out predictions on a daily basis for the same dependent variable. My intention is to assign weights to those two predictions and calculate a weighted ...
4
votes
1answer
1k views

Geometric and harmonic means in ensembling methods

When using ensembling methods for regression, a common approach is to average (using the arithmetic mean) the outputs of the weak learners in order to obtain the output of the ensemble. Is there a ...
-1
votes
1answer
90 views

Classification Ensemble with Random Forest as base classifier

I have created a classification ensemble with random forest as a base classifier. Each random forest has 500 trees. There are total 100 such forests in the ensemble. Majority voting is being used as ...
1
vote
0answers
60 views

AdaBoost - Ensemble model perform poor than best weak classifier

Can Adaboost's ensemble classifier perform worse than the best of the weak learners considered? If so when in what case of weak learner the ensemble learning does not perform better?
0
votes
3answers
4k views

How to apply ensemble clustering method?

I need to use ensemble clustering method by using python in my data set. I already applied k-means clustering by using scikit learn library. I also applied different classification method also find ...