Skip to main content
OverflowAI is here! AI power for your Stack Overflow for Teams knowledge community. Learn more
5 votes
Accepted

Are "Gradient Boosting Machines (GBM)" and GBDT exactly the same thing?

Boosting is an ensemble technique where predictors are ensembled sequentially one after the other(youtube tutorial. The term gradient of gradient boosting means that they are ensembled using the ...
Carlos Mougan's user avatar
5 votes
Accepted

How to apply Stacking cross validation for time-series data?

TL;DR Time-series algorithms assume that data points are ordered. Traditional K-Fold cannot be used for time series because it doesn't take into account the order in which data points appear. One ...
Bruno Lubascher's user avatar
4 votes
Accepted

Why Extra-trees should only be used within ensemble methods?

In a random forest tree, a random subset of features is available for consideration at each split. Extra-trees takes this a step further by using a random threshold at each split. The idea is that a ...
from keras import michael's user avatar
3 votes

How to apply Stacking cross validation for time-series data?

Standard TimeSeriesSplit from sklearn is not able to work with StackingRegressor because ...
Marco Cerliani's user avatar
3 votes

How to apply ensemble clustering method?

You might find the following libraries helpful: Python: Cluster Ensembles R: diceR
Alciomar Hollanda's user avatar
2 votes
Accepted

Geometric and harmonic means in ensembling methods

We can answer this overarching question by exploring a couple sub-questions: What are the properties of popular averaging formulae? Geometric: $ \space $ $ \space $ $ \begin{equation} \bigg(\...
Ben's user avatar
  • 2,572
2 votes
Accepted

Collection of several learners

Here's my two cents: Yes, if you want to perform enseble each model should be trained on the same set of data. The technique you described is stacking, because you "stack" each prediction and simply ...
RLave's user avatar
  • 278
2 votes
Accepted

What is the form of data used for prediction with generalized stacking ensemble?

Q1. This can be done either way. You may use only the base model predictions, or those and all the original features, or anywhere in between. Passing along the original features may be known as "...
Ben Reiniger's user avatar
  • 11.8k
2 votes
Accepted

Ensemble Techniques - Boosting

Actually, it depends on the boosting algorithm you used. In the original boosting algorithm (Schapire 1990), three classifiers are used (say $C_1$, $C_2$ and $C_3$). The training dataset is randomly ...
1tan Wang's user avatar
  • 358
2 votes
Accepted

Feed output of neural networks into other network in tensorflow

The concatenate function can connect 2 neural networks. You just have to be careful with the dimensions between them. Here is a pseudo-code: ...
Nicolas Martin's user avatar
2 votes
Accepted

How to train with cross validation? and which f1 score to choose?

I think you confused some technical names. Cross-Validation is the name of the procedure, and it has some techniques or approaches such as k-fold cross-validation, train test split, etc. All are ...
Shahriyar Mammadli's user avatar
1 vote
Accepted

Mixing categorical data and time-series data for regression purpose

There are multipe approaches you could try. Here are some (without the claim to completeness): You could try to extract meaningful / helpful features from the time-series. A start might be counts, ...
Broele's user avatar
  • 1,450
1 vote

How to boost the performance of a single decision tree by adding additional trees?

One option is cascading - putting machine learning models in a row where the output of one model becomes the input of another model. The first model is typically high recall, the second model is high ...
Brian Spiering's user avatar
1 vote
Accepted

In XGBoost, how is a leaf index corresponding to the particular leaf node in actual base learner trees?

I think what you are seeing is the fact that all nodes in the tree are indexed because a priori the model doesn't know where splits will happen (i.e. any node could be a leaf). My guess is that the ...
Brandon Donehoo's user avatar
1 vote

Found input variables with inconsistent numbers of samples: ValueError

In your 2nd last line, you are overwriting the variable x, which previously held your input X data.
GooJ's user avatar
  • 435
1 vote
Accepted

Why do I get an almost perfect fit as well as bias variance tradeoff with my time series forecast?

Without more details, it seems to me that you have a data leak problem. How did you split the data in train/test? Notice that you're dealing with a time-series problem, so the standard random split ...
alexmolas's user avatar
  • 556
1 vote

Feature Selection using Stacking Ensemble?

If you want to use a Stacking classifier I would recommend to change the feature selection algorithm to PermutationImportance, which is model agnostic way of computing importance based on random ...
Multivac's user avatar
  • 2,969
1 vote

How to assign a weight for classifiers when using weighted majority voting?

You can test each base classifier on a hold-out dataset and come up with a performance metric say accuracy for each model. You can then use each model's accuracy as a weight when combining predictions ...
Jayaram Iyer's user avatar
1 vote

Is there a closed formula/function for decision trees?

It can be expressed as a linear combination of indicators, where the weights are the training averages on each partition of the space, and the indicator is the indicator for that partition. Having ...
David Masip's user avatar
  • 6,081
1 vote

What if the votes for 2 classes are equal in an ensemble learning technique?

Technically RF in sklearn to get final prediction sums the probability values and divides by number of estimators. Refer to _accumulate_prediction function in the repo: ...
Abhishek Sawant's user avatar
1 vote

What if the votes for 2 classes are equal in an ensemble learning technique?

Depending on the implementation, this problem never occurs. Most of the implementations build an odd number of trees or models to ensure one class's dominance. However, some implementations allow an ...
Shahriyar Mammadli's user avatar
1 vote

Incorrect multi-variate anomaly detection - Isolation Forest Python

If you know beforehand the percentage of outliers present in your data, you should set the parameter contamination this will be the threshold used for the ...
Multivac's user avatar
  • 2,969
1 vote
Accepted

What is the difference between ensemble methods and hybrid methods, or is there none?

Here is a para that I found by searching What are hybrid methods in Machine Learning, on google. "In general, it is based on combining two different machine learning techniques. For example, a ...
Pushkaraj Joshi's user avatar
1 vote

Can we use boosting algorithms like Adaboost and gradient boosting with only one classifier

Boosting typically only use one algorithm as it's base learner (almost exclusively decision trees). However, you could use a mixed set of algorithms as your base learners. Something like this: ...
Simon Larsson's user avatar
1 vote
Accepted

How can I improve my model on a very very small dataset?

From what you say, I think you should start with checking three options: I) Ordinary least squares (OLS): Just run a „normal“ linear regression. This will not yield great predictions, but you could ...
Peter's user avatar
  • 7,516
1 vote
Accepted

Neural Network Multiple | Averging predictions

There are multiple ways you could do. All of them are categorized under Ensemble methods in machine learning. Voting classifiers: which is the simplest way. You just take votes based on the label ...
aminrd's user avatar
  • 340
1 vote

Ensemble Techniques - Bagging | Subset size

like said in a previous answer, the exact subsample parameter value depends on your data. But a usual starting parameter that gets you good results in general, and doesn't hurt the data distribution ...
Blenz's user avatar
  • 2,074
1 vote
Accepted

Difference between bagging and boosting

Bagging: Also known as Bootstrap Aggregation is an ensemble method. First, we create random samples of the training data set (sub sets of training data set). Then, we build a classifier for each ...
CodeMaster GoGo's user avatar
1 vote

How to construct a self learning process for an ensemble model?

It is not much more sofisticated than what you thought, but you can try training a linear regression (in the online setting, by stochastic gradient descent) whose inputs are the predictions spit by ...
David Masip's user avatar
  • 6,081
1 vote

How to apply ensemble clustering method?

As far as I know, scikit-learn has no library for ensemble clustering. On the other hand, you can apply the method on your dataset as follows: ...
gagagio's user avatar
  • 11

Only top scored, non community-wiki answers of a minimum length are eligible