7 votes

Is it possible to build ensemble models without a decision tree?

all the ensemble models I came through so far use/described using the decision tree. Random Forest is the "ensemble version" of decision trees. It's a commonly used ensemble method because ...
Erwan's user avatar
  • 25.3k
4 votes
Accepted

What is the difference between horizontal and vertical ensemble?

I think the most highly-referenced source for these terms is Horizontal and Vertical Ensemble with Deep Representation for Classification (Xe, Xu, Chuang 2013). That would be the best place to get a ...
Savage Henry's user avatar
4 votes

Bagging vs pasting in ensemble learning

Let's say we have a set of 40 numbers from 1 to 40. We have to pick 4 subsets of 10 numbers. Case 1 - Bagging - We will pick the first number, put it back, and then pick the next. This makes all the ...
10xAI's user avatar
  • 5,584
3 votes
Accepted

Can clustering my data first help me learn better classifiers?

It is absolutely a way to improve your classifier's accuracy. Actually a "strong" enough classifier such as a neural network could be able to learn by itself these clusters. However, you would need a ...
Jonathan DEKHTIAR's user avatar
3 votes

Combining Classifiers with different Precision and Recall values

You want to ensemble your two algorithms. The way to do that is to not just use e.g. sklearn's precision and recall metric functions, but to actually obtain probabilities for being positive (most ...
Chris Ivan's user avatar
2 votes

xgboost cannot identify perfectly fitting regression line

I think that the reason for this to happen is that tree-based methods have problems with linear problems. This is because tree-based methods do partitions of the variables, and not on combinations of ...
David Masip's user avatar
  • 6,051
2 votes
Accepted

Geometric and harmonic means in ensembling methods

We can answer this overarching question by exploring a couple sub-questions: What are the properties of popular averaging formulae? Geometric: $ \space $ $ \space $ $ \begin{equation} \bigg(\...
Ben's user avatar
  • 2,562
2 votes
Accepted

Handling Categorical Features on NGBoost

It does not support at the time (it will come just as xgboost did not have to have it) Given thats its a boosting method in the first place one can ask whats the history of xbgoost and subsequent cat ...
Noah Weber's user avatar
  • 5,669
2 votes

Is it possible to build ensemble models without a decision tree?

At the highest level of abstraction, the answer is yes. You can send a set of values to be scored to every model in an ensemble, and then combine the resulting of set of scores into a single score ...
NaiveBayesian's user avatar
2 votes
Accepted

How to train with cross validation? and which f1 score to choose?

I think you confused some technical names. Cross-Validation is the name of the procedure, and it has some techniques or approaches such as k-fold cross-validation, train test split, etc. All are ...
Shahriyar Mammadli's user avatar
2 votes
Accepted

Ensemble method with Blackbox Classifiers

After some more reviewing I came across the following solution: So basically Logistic regression's input is supposed to be numerical values, while the Neural networks (NNs) that I have gave string-...
user_04248753498's user avatar
1 vote

Is it possible to build ensemble models without a decision tree?

You can indeed use other weak learners (as the components of an ensemble are commonly called) than just decision trees. That said, decision tree ensembles are most widely used, especially gradient ...
MightyCurious's user avatar
1 vote

Can i use other regression types that arent based in decision trees to use it like a weak learners in gradient boosting?

The issue of using any linear model (a polynomial regression is a particular case of a linear model, with polynomial features), is that an ensemble of linear models is still a linear model. So, the ...
David Masip's user avatar
  • 6,051
1 vote

"ValueError: Data cardinality is ambiguous" in model ensemble with 2 different inputs

You can not feed your network with two inputs with a different number of samples, and this also does not make sense. You have 2 inputs with shapes (502,) and ...
Kaveh's user avatar
  • 174
1 vote

How to assign a weight for classifiers when using weighted majority voting?

You can test each base classifier on a hold-out dataset and come up with a performance metric say accuracy for each model. You can then use each model's accuracy as a weight when combining predictions ...
Jayaram Iyer's user avatar
1 vote

What are some good models to complement XGBOOST in stacking?

It really depends on how other models behave on your particular dataset, but generally, you should check 2 things: Predictions from ensembled models are not highly correlated and other models also ...
Yaroslaw Homenko's user avatar
1 vote

How do I combine predictions from classifiers for two different problem?

My intuition would be to try to integrate the information about the products directly in the original model. Typically the possible products in a shipment can be represented as boolean features (one ...
Erwan's user avatar
  • 25.3k
1 vote

Ensembling expressions

You can make the same question with every Machine Learning algorithm, and still the answer will remain very similar. What's the advantage of Linear regression over Decision Trees? To answer this you ...
Carlos Mougan's user avatar
1 vote

Ensembling expressions

That is an empirical question. The answer will change for different models and different datasets. The best approach would use cross validation to see which ensembling technique has the best score on ...
Brian Spiering's user avatar
1 vote

Visualizing F-score differences in information extraction

Yes, I think that's a sound approach and a good way to compare different systems. A ROC curve comparison is usually more informative than the raw performance scores, but it's still quite general. In ...
Erwan's user avatar
  • 25.3k
1 vote

grid search - optimal weighting of classifiers

GridSearch finds those optimals weights for you. You can access these weights through the attribute best_params_ of the GridSearch object, which will return all the optimal parameters (including the ...
θ Grunberg's user avatar
1 vote

How can the Adaboost technique be called an ensemble learning technique?

AdaBoost or Adaptive Boost is a boosting ensemble model which works by learning from it's previous mistakes, ie: misclassified data points. We specify the number of decision trees to be generated ...
Raghav Kukreti's user avatar
1 vote

How can I make ROC and compute AUC?

Is an interesting question. You are solving a optimization problem $max$ $AUC(\alpha_1*X_1 + \alpha_2*X_2,Y)$ $s.t.$ $\sum^2_{i=1}\alpha = 1$ You are maximizing the AUC using the models 1 and 2 (...
Juan Esteban de la Calle's user avatar
1 vote

Combining Classifiers with different Precision and Recall values

It seems like your goal is to improve the performance of your model using the outputs of two existing models with different strengths. The specifics of how you might want to combine these two models ...
NiX's user avatar
  • 11
1 vote

When and how to use bagging?

Bagging main goal is to minimize variance of your model. Basically, if you have a model that is on average pretty accurate but inconsistent (meaning, it does well for a given data set, poorly ...
Tophat's user avatar
  • 2,420
1 vote

How to visualize Ensemble Models ( Random Forest) with 1000 estimators

You can try SHAP which visually explains the output of (many) machine learning model(s) including LightGBM and XGBoost. However, please note that it will not give you the entire Ensemble Model (Trees)...
Khurram Majeed's user avatar
1 vote

xgboost cannot identify perfectly fitting regression line

(adding to what's said above by @David), The short answer is that, You can't expect the tree based models to Extrapolate... Had asked on Slack (quoting miguel_perez)and this was the reply, realize ...
Aditya's user avatar
  • 2,470
1 vote

Methods for ensembling ranked lists?

In this case, you might want to convert these into pair-wise relationships (e.g. item1 < item3), put together what you get from the different methods, and find a ranking which agrees with them the ...
anymous.asker's user avatar
1 vote

How to create an ensemble that gives precedence to a specific classifier

Based on your description, it looks like different models have different biases. Two important questions: do you have any data imbalance problem? What kind of models you are using? Using stacking ...
Bashar Haddad's user avatar
1 vote

How to create an ensemble that gives precedence to a specific classifier

Feature-Weighted Linear Stacking might be what you are looking for. FWLS combines model predictions linearly using coefficients that are themselves linear functions of meta-features. In your ...
Imran's user avatar
  • 2,381

Only top scored, non community-wiki answers of a minimum length are eligible