5
votes
Accepted
Are "Gradient Boosting Machines (GBM)" and GBDT exactly the same thing?
Boosting is an ensemble technique where predictors are ensembled sequentially one after the other(youtube tutorial. The term gradient of gradient boosting means that they are ensembled using the ...
- 6,332
5
votes
Accepted
How to apply Stacking cross validation for time-series data?
TL;DR
Time-series algorithms assume that data points are ordered.
Traditional K-Fold cannot be used for time series because it doesn't take into account the order in which data points appear.
One ...
- 3,578
4
votes
Accepted
Why Extra-trees should only be used within ensemble methods?
In a random forest tree, a random subset of features is available for consideration at each split. Extra-trees takes this a step further by using a random threshold at each split. The idea is that a ...
3
votes
How to apply Stacking cross validation for time-series data?
Standard TimeSeriesSplit from sklearn is not able to work with StackingRegressor because ...
- 186
3
votes
How to apply ensemble clustering method?
You might find the following libraries helpful:
Python: Cluster Ensembles
R: diceR
2
votes
Accepted
Geometric and harmonic means in ensembling methods
We can answer this overarching question by exploring a couple sub-questions:
What are the properties of popular averaging formulae?
Geometric:
$ \space $
$ \space $
$ \begin{equation}
\bigg(\...
- 2,582
2
votes
Accepted
Collection of several learners
Here's my two cents:
Yes, if you want to perform enseble each model should be trained on the same set of data. The technique you described is stacking, because you "stack" each prediction and simply ...
- 278
2
votes
Accepted
What is the form of data used for prediction with generalized stacking ensemble?
Q1. This can be done either way. You may use only the base model predictions, or those and all the original features, or anywhere in between. Passing along the original features may be known as "...
- 12k
2
votes
Accepted
Ensemble Techniques - Boosting
Actually, it depends on the boosting algorithm you used.
In the original boosting algorithm (Schapire 1990), three classifiers are used (say $C_1$, $C_2$ and $C_3$). The training dataset is randomly ...
- 358
2
votes
Accepted
Feed output of neural networks into other network in tensorflow
The concatenate function can connect 2 neural networks. You just have to be careful with the dimensions between them.
Here is a pseudo-code:
...
- 4,814
2
votes
Accepted
How to train with cross validation? and which f1 score to choose?
I think you confused some technical names. Cross-Validation is the name of the procedure, and it has some techniques or approaches such as k-fold cross-validation, train test split, etc. All are ...
- 1,168
1
vote
Accepted
Mixing categorical data and time-series data for regression purpose
There are multipe approaches you could try. Here are some (without the claim to completeness):
You could try to extract meaningful / helpful features from the time-series. A start might be counts, ...
- 1,717
1
vote
How to boost the performance of a single decision tree by adding additional trees?
One option is cascading - putting machine learning models in a row where the output of one model becomes the input of another model. The first model is typically high recall, the second model is high ...
- 22.1k
1
vote
Accepted
In XGBoost, how is a leaf index corresponding to the particular leaf node in actual base learner trees?
I think what you are seeing is the fact that all nodes in the tree are indexed because a priori the model doesn't know where splits will happen (i.e. any node could be a leaf). My guess is that the ...
- 376
1
vote
Found input variables with inconsistent numbers of samples: ValueError
In your 2nd last line, you are overwriting the variable x, which previously held your input X data.
- 445
1
vote
Accepted
Why do I get an almost perfect fit as well as bias variance tradeoff with my time series forecast?
Without more details, it seems to me that you have a data leak problem.
How did you split the data in train/test? Notice that you're dealing with a time-series problem, so the standard random split ...
- 566
1
vote
Feature Selection using Stacking Ensemble?
If you want to use a Stacking classifier I would recommend to change the feature selection algorithm to PermutationImportance, which is model agnostic way of computing importance based on random ...
- 3,009
1
vote
How to assign a weight for classifiers when using weighted majority voting?
You can test each base classifier on a hold-out dataset and come up with a performance metric say accuracy for each model. You can then use each model's accuracy as a weight when combining predictions ...
- 825
1
vote
Is there a closed formula/function for decision trees?
It can be expressed as a linear combination of indicators, where the weights are the training averages on each partition of the space, and the indicator is the indicator for that partition.
Having ...
- 6,111
1
vote
What if the votes for 2 classes are equal in an ensemble learning technique?
Technically RF in sklearn to get final prediction sums the probability values and divides by number of estimators.
Refer to _accumulate_prediction function in the repo:
...
- 111
1
vote
What if the votes for 2 classes are equal in an ensemble learning technique?
Depending on the implementation, this problem never occurs. Most of the implementations build an odd number of trees or models to ensure one class's dominance.
However, some implementations allow an ...
- 1,168
1
vote
Incorrect multi-variate anomaly detection - Isolation Forest Python
If you know beforehand the percentage of outliers present in your data, you should set the parameter contamination this will be the threshold used for the ...
- 3,009
1
vote
Accepted
What is the difference between ensemble methods and hybrid methods, or is there none?
Here is a para that I found by searching What are hybrid methods in Machine Learning, on google.
"In general, it is based on combining two different machine learning techniques. For example, a ...
- 733
1
vote
Can we use boosting algorithms like Adaboost and gradient boosting with only one classifier
Boosting typically only use one algorithm as it's base learner (almost exclusively decision trees). However, you could use a mixed set of algorithms as your base learners.
Something like this:
...
- 4,243
1
vote
Accepted
How can I improve my model on a very very small dataset?
From what you say, I think you should start with checking three options:
I) Ordinary least squares (OLS): Just run a „normal“ linear regression. This will not yield great predictions, but you could ...
- 7,596
1
vote
Accepted
Neural Network Multiple | Averging predictions
There are multiple ways you could do. All of them are categorized under Ensemble methods in machine learning.
Voting classifiers: which is the simplest way. You just take votes based on the label ...
- 340
1
vote
Ensemble Techniques - Bagging | Subset size
like said in a previous answer, the exact subsample parameter value depends on your data.
But a usual starting parameter that gets you good results in general, and doesn't hurt the data distribution ...
- 2,094
1
vote
Accepted
Difference between bagging and boosting
Bagging: Also known as Bootstrap Aggregation is an ensemble method. First, we create random samples of the training data set (sub sets of training data set). Then, we build a classifier for each ...
- 788
1
vote
How to construct a self learning process for an ensemble model?
It is not much more sofisticated than what you thought, but you can try training a linear regression (in the online setting, by stochastic gradient descent) whose inputs are the predictions spit by ...
- 6,111
1
vote
How to apply ensemble clustering method?
As far as I know, scikit-learn has no library for ensemble clustering. On the other hand, you can apply the method on your dataset as follows:
...
- 11
Only top scored, non community-wiki answers of a minimum length are eligible