Questions tagged [ensemble-learning]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
15 views

Is it possible to extract precise decision boundaries from a random forest for a multiclass classification?

I have a random forest (argmax post-processor) with 3 trees and 10 input features. The final outcome of the random forest is either true or false depending on the combination of the feature values. Is ...
user avatar
0 votes
0 answers
9 views

Why would an ensemble model perform worse than all individual models? biomod2

I'm using biomod2 in R and my ensemble model performs worse on the evaluation data (drastically lower ROC, 0.835) than any of the individual models (ROC ranges 0.89-0.97). What could be causing this? ...
user avatar
0 votes
1 answer
34 views

Why do I get an almost perfect fit as well as bias variance tradeoff with my time series forecast?

In order to achieve scalable and robust time series forecast models, I am currently experimenting with metalearner ensembles. Note, that I am also using a global modeling approach, so all time series ...
user avatar
1 vote
0 answers
37 views

Can numerical encoding really replace one-hot encoding?

I am reading these articles (see below), which advocate the use of numerical encoding rather than one hot encoding for better interpretability of feature importance output from ensemble models. This ...
user avatar
0 votes
1 answer
176 views

Feature Selection using Stacking Ensemble?

I want to combine some estimators, such as Logistic Regression, Gaussian NB and ...
user avatar
  • 45
1 vote
1 answer
41 views

How to assign a weight for classifiers when using weighted majority voting?

I am trying to apply weighted majority voting on an ensemble as a combiner method. I read different papers and articles, however, I am still a bit lost on: How the weighted majority voting works How ...
user avatar
  • 53
1 vote
0 answers
20 views

Ensemble of different reservoirs (echo state networks)

Suppose I want to do reservoir computing to classify the input to the proper category (e.g. recognizing a handwritten letter). Ideally, after training a single reservoir and testing it, there would be ...
user avatar
  • 111
0 votes
1 answer
69 views

Is there a closed formula/function for decision trees?

i've been studying gradient boosting so realize the pure algorithm requires a function F/model to get boosted.What is the explicit F on gradient boosting trees?
user avatar
0 votes
2 answers
106 views

What if the votes for 2 classes are equal in an ensemble learning technique?

Suppose in ensemble learning technique, if the number of models that predict class 1 is equal to the number of models that predict class 0. Then, which class will be decided as output?
user avatar
  • 21
0 votes
1 answer
409 views

How to train with cross validation? and which f1 score to choose?

I got similar results in 2 models which consists of similar algorithms. Model 1 with cv=10 has a f1'micro' of 0.941. See code below. Model 2 only train test split (no cv) has f1'micro' 0.953. Now here ...
user avatar
  • 329
1 vote
1 answer
392 views

In XGBoost, how is a leaf index corresponding to the particular leaf node in actual base learner trees?

I've trained a XGBoost model for regression, where the max depth is 2. ...
user avatar
3 votes
1 answer
655 views

Are "Gradient Boosting Machines (GBM)" and GBDT exactly the same thing?

In the category of Gradient Boosting, I find some terms confusing. I'm aware that XGBoost includes some optimization in comparison to conventional Gradient Boosting. But are Gradient Boosting ...
user avatar
1 vote
0 answers
131 views

Stacking - Appropriate base and meta models

When implementing stacking for model building and prediction (For example using sklearn's StackingRegressor function) what is the appropriate choice of models for the base models and final meta model?...
user avatar
0 votes
2 answers
73 views

Incorrect multi-variate anomaly detection - Isolation Forest Python

My data looks like below. it has 333 rows and 2 columns. Clearly the first row is anomaly. ndf: ...
user avatar
  • 111
1 vote
1 answer
2k views

What is the difference between ensemble methods and hybrid methods, or is there none?

I have the feeling that these terms often are used as synonyms for one another, however they have the same goal, namely increasing prediction accuracy by combining different algorithms. My question ...
user avatar
1 vote
1 answer
40 views

What is the form of data used for prediction with generalized stacking ensemble?

I am very confused as to how training data is split and on what data level 0 predictions are made when using generalized stacking. This question is similar to mine, but the answer is not sufficiently ...
user avatar
2 votes
1 answer
153 views

How can I improve my model on a very very small dataset?

I am starting as a PhD student and we want to find appropriate materials (with certain qualities) from basic chemical properties like charge, etc. There are a lot of models and datasets in similar ...
user avatar
  • 23
1 vote
0 answers
3k views

ValueError: Graph disconnected: cannot obtain value for tensor Tensor

I'm trying to perform a stacking ensemble of three VGG-16 models, all custom-trained on my personal dataset and having the same input shape. This is the code: ...
user avatar
  • 311
2 votes
1 answer
36 views

Neural Network Multiple | Averging predictions

I am training multiple neural networks with various parameters. I am trying to average their predictions, but I am not really sure what that means, I am confused about what to average exactly. Here is ...
user avatar
  • 257
1 vote
2 answers
149 views

Ensemble Techniques - Bagging | Subset size

I do have a question on ensemble techniques Baggging/Boosting. - What would be the subset size for Bagging?
user avatar
  • 234
0 votes
1 answer
39 views

Ensemble Techniques - Boosting

I understand boosting is a sequential learning technique and it use the prediction from previous model as a dataset for new model ,after adding weight to the misclassified data points. The point ...
user avatar
  • 234
1 vote
1 answer
250 views

Bagging with Neural Networks Best practices

I am trying to build a majority vote system for 3 Neural Networks, and I came across the concept of Bagging method. Actually, I want to use neural networks as weak learners (I know it's debatable, but ...
user avatar
  • 257
1 vote
0 answers
38 views

How to choose learning models for different levels of a Stacking Ensemble?

I was wondering if there is any specific rule for designing the architecture of a Stacking Ensemble (e.g. number of levels, models in each level, and models in general). Also, can there be an overlap ...
user avatar
  • 571
-1 votes
1 answer
36 views

If There is a case where decision trees are getting overfitted so by using gradient boost method do we solve that problem?

I have came across a case where my decision trees are getting overfitting so by using methods like gradient boost can I solve that problem.
user avatar
1 vote
1 answer
155 views

Can we use boosting algorithms like Adaboost and gradient boosting with only one classifier

I have been working on ensemble learning and I came across this doubt that unlike other ensemble learning algorithms like voting classifier a can we only use one classifier with boosting.
user avatar
1 vote
0 answers
41 views

Ensemble technique for combining predictions from classification and regression algorithms

Given an anomaly detection problem - A, I have divided the problem into two independent subtasks -A1 and ...
user avatar
  • 121
1 vote
0 answers
14 views

Ensemble learning for multiple hypothesis classes

Just to confirm if the following description falls in the category of ensemble learning. Suppose given a training set $D=\{(X,Y)\}$ we are asked to train a regressor. But now the way we do it is to ...
user avatar
  • 11
4 votes
2 answers
2k views

How to apply Stacking cross validation for time-series data?

Normally stacking algorithm uses K-fold cross validation technique to predict oof validation that used for level 2 prediction. In case of time-series data (say stock movement prediction), K-fold ...
user avatar
1 vote
1 answer
1k views

Difference between bagging and boosting

Can anyone explain me the basic difference between bagging and boosting and which technique can be used in which scenario?
user avatar
2 votes
1 answer
273 views

Why Extra-trees should only be used within ensemble methods?

I was reading scikit-learn documentation for Extremely Randomized Trees and I found this warning: Warning: Extra-trees should only be used within ensemble methods. Why is that?
user avatar
0 votes
1 answer
67 views

Collection of several learners [closed]

I have few questions for which I could not extract answers from text books and online tutorials. Therefore, will be extremely grateful if the following points are clarified. 1) If I want to apply SVM,...
user avatar
3 votes
0 answers
91 views

Stacking when the the target variable is categorical?

I'm trying to use stacking when predicting for the infamous Iris dataset. Also, I'd like to build to stacked classifier by myself which means I don't want to use mlxtend because it's too "easy" and ...
user avatar
1 vote
1 answer
77 views

How to construct a self learning process for an ensemble model?

I have two different (regression) models spitting out predictions on a daily basis for the same dependent variable. My intention is to assign weights to those two predictions and calculate a weighted ...
user avatar
4 votes
1 answer
1k views

Geometric and harmonic means in ensembling methods

When using ensembling methods for regression, a common approach is to average (using the arithmetic mean) the outputs of the weak learners in order to obtain the output of the ensemble. Is there a ...
user avatar
  • 5,674
-1 votes
1 answer
96 views

Classification Ensemble with Random Forest as base classifier

I have created a classification ensemble with random forest as a base classifier. Each random forest has 500 trees. There are total 100 such forests in the ensemble. Majority voting is being used as ...
user avatar
1 vote
0 answers
61 views

AdaBoost - Ensemble model perform poor than best weak classifier

Can Adaboost's ensemble classifier perform worse than the best of the weak learners considered? If so when in what case of weak learner the ensemble learning does not perform better?
user avatar
0 votes
3 answers
4k views

How to apply ensemble clustering method?

I need to use ensemble clustering method by using python in my data set. I already applied k-means clustering by using scikit learn library. I also applied different classification method also find ...
user avatar
  • 295