Questions tagged [ensemble-learning]
The ensemble-learning tag has no usage guidance.
44
questions
1
vote
1
answer
33
views
How does the stacking works?
Suppose that I have $n$ trained weak base models: $m_1, m_2, ..., m_n$
As I understand after training that models we get their predictions on validation dataset, let's consider single element of ...
0
votes
0
answers
20
views
Can you please provide one fully solved gradient boosting regression numerical example (not python code) [duplicate]
Can you please provide one fully solved gradient boost regression numerical example (not Python code).
1
vote
1
answer
111
views
Mixing categorical data and time-series data for regression purpose
I came across a problem and I have been looking on internet how to solve it without finding a solution that fits my need. I am trying to predict investors behavior. To be precise, I would like to ...
- 13
0
votes
0
answers
49
views
The weighted average ensemble model does not train on the whole data
I am using custom data generator. I want to apply weighted average ensemble. The training set has 1042 samples, and validation indicates 298 samples. The batch size is 64.
when I run this :
...
- 1
2
votes
2
answers
72
views
How to boost the performance of a single decision tree by adding additional trees?
I have a binary classification task and the data has imbalance issue (99% is negative and 1% is positive). I am able to build a decision tree that is carefully tuned, weighted, and post-pruned. Take ...
- 21
0
votes
0
answers
35
views
What is the formula to combine N correlated classifiers into single optimal one?
As we know if we train N probabilistic classifiers on same dataset, they will have some degree of correlation. As we also know, there is some method to assign optimal coefficients/factors/weights to ...
1
vote
1
answer
110
views
Feed output of neural networks into other network in tensorflow
I have two well trained neural networks, shown as:
...
0
votes
1
answer
871
views
Found input variables with inconsistent numbers of samples: ValueError
Today I am trying build ensemble model. Where I am working with iris dataset. In my model I am using ...
0
votes
0
answers
41
views
Is it possible to extract precise decision boundaries from a random forest for a multiclass classification?
I have a random forest (argmax post-processor) with 3 trees and 10 input features. The final outcome of the random forest is either true or false depending on the combination of the feature values. Is ...
0
votes
1
answer
55
views
Why do I get an almost perfect fit as well as bias variance tradeoff with my time series forecast?
In order to achieve scalable and robust time series forecast models, I am currently experimenting with metalearner ensembles.
Note, that I am also using a global modeling approach, so all time series ...
- 145
1
vote
0
answers
71
views
Can numerical encoding really replace one-hot encoding?
I am reading these articles (see below), which advocate the use of numerical encoding rather than one hot encoding for better interpretability of feature importance output from ensemble models. This ...
- 41
0
votes
1
answer
660
views
Feature Selection using Stacking Ensemble?
I want to combine some estimators, such as Logistic Regression, Gaussian NB and ...
- 45
1
vote
1
answer
154
views
How to assign a weight for classifiers when using weighted majority voting?
I am trying to apply weighted majority voting on an ensemble as a combiner method. I read different papers and articles, however, I am still a bit lost on:
How the weighted majority voting works
How ...
- 53
1
vote
0
answers
24
views
Ensemble of different reservoirs (echo state networks)
Suppose I want to do reservoir computing to classify the input to the proper category (e.g. recognizing a handwritten letter).
Ideally, after training a single reservoir and testing it, there would be ...
- 111
0
votes
1
answer
117
views
Is there a closed formula/function for decision trees?
i've been studying gradient boosting so realize the pure algorithm requires a function F/model to get boosted.What is the explicit F on gradient boosting trees?
- 29
0
votes
2
answers
326
views
What if the votes for 2 classes are equal in an ensemble learning technique?
Suppose in ensemble learning technique, if the number of models that predict class 1 is equal to the number of models that predict class 0. Then, which class will be decided as output?
- 21
0
votes
1
answer
578
views
How to train with cross validation? and which f1 score to choose?
I got similar results in 2 models which consists of similar algorithms.
Model 1 with cv=10 has a f1'micro' of 0.941. See code below.
Model 2 only train test split (no cv) has f1'micro' 0.953.
Now here ...
- 329
1
vote
1
answer
761
views
In XGBoost, how is a leaf index corresponding to the particular leaf node in actual base learner trees?
I've trained a XGBoost model for regression, where the max depth is 2.
...
- 392
4
votes
1
answer
1k
views
Are "Gradient Boosting Machines (GBM)" and GBDT exactly the same thing?
In the category of Gradient Boosting, I find some terms confusing.
I'm aware that XGBoost includes some optimization in comparison to conventional Gradient Boosting.
But are Gradient Boosting ...
- 392
1
vote
0
answers
199
views
Stacking - Appropriate base and meta models
When implementing stacking for model building and prediction (For example using sklearn's StackingRegressor function) what is the appropriate choice of models for the base models and final meta model?...
- 735
0
votes
2
answers
95
views
Incorrect multi-variate anomaly detection - Isolation Forest Python
My data looks like below. it has 333 rows and 2 columns. Clearly the first row is anomaly.
ndf:
...
- 111
1
vote
2
answers
4k
views
What is the difference between ensemble methods and hybrid methods, or is there none?
I have the feeling that these terms often are used as synonyms for one another, however they have the same goal, namely increasing prediction accuracy by combining different algorithms.
My question ...
- 163
1
vote
1
answer
66
views
What is the form of data used for prediction with generalized stacking ensemble?
I am very confused as to how training data is split and on what data level 0 predictions are made when using generalized stacking. This question is similar to mine, but the answer is not sufficiently ...
- 309
2
votes
1
answer
311
views
How can I improve my model on a very very small dataset?
I am starting as a PhD student and we want to find appropriate materials (with certain qualities) from basic chemical properties like charge, etc. There are a lot of models and datasets in similar ...
- 23
1
vote
0
answers
4k
views
ValueError: Graph disconnected: cannot obtain value for tensor Tensor
I'm trying to perform a stacking ensemble of three VGG-16 models, all custom-trained on my personal dataset and having the same input shape. This is the code:
...
- 311
2
votes
1
answer
39
views
Neural Network Multiple | Averging predictions
I am training multiple neural networks with various parameters. I am trying to average their predictions, but I am not really sure what that means, I am confused about what to average exactly. Here is ...
- 257
1
vote
2
answers
198
views
Ensemble Techniques - Bagging | Subset size
I do have a question on ensemble techniques Baggging/Boosting.
- What would be the subset size for Bagging?
- 234
0
votes
1
answer
45
views
Ensemble Techniques - Boosting
I understand boosting is a sequential learning technique and it use the prediction from previous model as a dataset for new model ,after adding weight to the misclassified data points.
The point ...
- 234
1
vote
1
answer
355
views
Bagging with Neural Networks Best practices
I am trying to build a majority vote system for 3 Neural Networks, and I came across the concept of Bagging method. Actually, I want to use neural networks as weak learners (I know it's debatable, but ...
- 257
1
vote
0
answers
43
views
How to choose learning models for different levels of a Stacking Ensemble?
I was wondering if there is any specific rule for designing the architecture of a Stacking Ensemble (e.g. number of levels, models in each level, and models in general). Also, can there be an overlap ...
- 611
-1
votes
1
answer
41
views
If There is a case where decision trees are getting overfitted so by using gradient boost method do we solve that problem?
I have came across a case where my decision trees are getting overfitting so by using methods like gradient boost can I solve that problem.
- 45
1
vote
1
answer
218
views
Can we use boosting algorithms like Adaboost and gradient boosting with only one classifier
I have been working on ensemble learning and I came across this doubt that unlike other ensemble learning algorithms like voting classifier a can we only use one classifier with boosting.
- 45
1
vote
0
answers
42
views
Ensemble technique for combining predictions from classification and regression algorithms
Given an anomaly detection problem - A, I have divided the problem into two independent subtasks -A1 and ...
- 121
1
vote
0
answers
17
views
Ensemble learning for multiple hypothesis classes
Just to confirm if the following description falls in the category of ensemble learning. Suppose given a training set $D=\{(X,Y)\}$ we are asked to train a regressor. But now the way we do it is to ...
- 11
5
votes
2
answers
3k
views
How to apply Stacking cross validation for time-series data?
Normally stacking algorithm uses K-fold cross validation technique to predict oof validation that used for level 2 prediction.
In case of time-series data (say stock movement prediction), K-fold ...
1
vote
1
answer
1k
views
Difference between bagging and boosting
Can anyone explain me the basic difference between bagging and boosting and which technique can be used in which scenario?
- 13
2
votes
1
answer
307
views
Why Extra-trees should only be used within ensemble methods?
I was reading scikit-learn documentation for Extremely Randomized Trees and I found this warning:
Warning: Extra-trees should only be used within ensemble methods.
Why is that?
- 221
0
votes
1
answer
69
views
Collection of several learners [closed]
I have few questions for which I could not extract answers from text books and online tutorials. Therefore, will be extremely grateful if the following points are clarified.
1) If I want to apply SVM,...
- 115
3
votes
0
answers
121
views
Stacking when the the target variable is categorical?
I'm trying to use stacking when predicting for the infamous Iris dataset. Also, I'd like to build to stacked classifier by myself which means I don't want to use mlxtend because it's too "easy" and ...
- 265
1
vote
1
answer
83
views
How to construct a self learning process for an ensemble model?
I have two different (regression) models spitting out predictions on a daily basis for the same dependent variable. My intention is to assign weights to those two predictions and calculate a weighted ...
- 153
4
votes
1
answer
1k
views
Geometric and harmonic means in ensembling methods
When using ensembling methods for regression, a common approach is to average (using the arithmetic mean) the outputs of the weak learners in order to obtain the output of the ensemble. Is there a ...
- 6,031
-1
votes
1
answer
117
views
Classification Ensemble with Random Forest as base classifier
I have created a classification ensemble with random forest as a base classifier. Each random forest has 500 trees. There are total 100 such forests in the ensemble. Majority voting is being used as ...
- 41
1
vote
0
answers
63
views
AdaBoost - Ensemble model perform poor than best weak classifier
Can Adaboost's ensemble classifier perform worse than the best of the weak learners considered? If so when in what case of weak learner the ensemble learning does not perform better?
- 83
0
votes
3
answers
5k
views
How to apply ensemble clustering method?
I need to use ensemble clustering method by using python in my data set. I already applied k-means clustering by using scikit learn library. I also applied different classification method also find ...
- 295