Questions tagged [ensemble-modeling]

In machine learning, ensemble methods combine multiple algorithms to make a prediction. Bagging, boosting, and stacking, are some examples.

Filter by
Sorted by
Tagged with
1
vote
0answers
6 views

Is there any way to plot ROC curve for Ensemble hard voting classifier?

I am working on a multi-class text classification problem and performing an Ensemble learning for text classification. I chose hard voting as ensemble technique. I tried to plot ROC curve for my ...
-1
votes
0answers
7 views

What is blending?

Many internet sides describe blending as "Blending is a word introduced by the Netflix winners"(in 2009 of the 1 mio $ Netflix competition) and it´s a method where you create a small holdout ...
1
vote
0answers
15 views

Decision Boundary with random new observations vs observations from test set

I'm trying to plot decision boundary for Decision Tree classifier. Classifier is trained on training set, and decision boundary (contour) using random new observations and observations from test set ...
3
votes
1answer
25 views

Classification model accuracy with ensemble methods

I came through this statement in a Machine Learning text book based on law of large numbers: Suppose you build an ensemble containing 1,000 classifiers that are individually correct only 51% of the ...
1
vote
1answer
28 views

Stacked Model performance?

I am currently working with a dataset that seems very easily separable and I have an accuracy of 99% for SVM (NN-98%, RF-98%, DT-96-97% and I have checked for leakage & overfitting). As part of my ...
4
votes
2answers
88 views

Stacking and Ensembling methods in Data Science

I understand that using stacking and ensembling has become popular, and these methods can give better results than using a single algorithm. My question is: What are the reasons, statistical or ...
-1
votes
0answers
13 views

Orange Data Mining: Using Image to predict inventories to purchase [closed]

Say we have two data sources, (1) a set of 200 images of random generic products (mugs, cups, scissors etc.) and that's processed by Orange via the image Embed function (SqueezeNet-Local) (2) a .csv ...
0
votes
1answer
28 views

XGBoost Log Loss different from GridSearchCV Log Loss

I have a classification problem where i am trying to predict if the data returns a 1 or 0. So you're classic binary classification. I have my set of data that I have split into the dependent variables ...
1
vote
1answer
26 views

How does bagging help reduce the variance

I learned that bagging helps reduce variance by averaging but I couldn't understand this. Can someone explain this intuitively?
0
votes
1answer
30 views

How do I Combine two CNN models (h5 format)?

I have two CNN models, both of them are trained on the same dataset. How do I combine/ensemble both to make predictions on test data? ...
1
vote
1answer
40 views

What is the difference between ensemble methods and hybrid methods, or is there none?

I have the feeling that these terms often are used as synonyms for one another, however they have the same goal, namely increasing prediction accuracy by combining different algorithms. My question ...
0
votes
0answers
8 views

Ensemble Model to Handle Different Image Attributes

I'm working on a project where I have images annotated across several attributes, say X, Y, Z...
0
votes
0answers
9 views

How to compare ensemble spread with RMSE in order to find data points not included in the interpolation region?

I am currently working on the refinement of a huge dataset. The concept is that I want to reduce as many training data as possible. I have generated more than 1 million data without labels. To give ...
1
vote
1answer
19 views

Model stacking with instance attributes [duplicate]

From what I have understood about model stacking: the meta estimator trains to combine the N-models predictions to fit the ground truth. Once trained, it combines the 1st level output to approach the ...
1
vote
1answer
17 views

What is the form of data used for prediction with generalized stacking ensemble?

I am very confused as to how training data is split and on what data level 0 predictions are made when using generalized stacking. This question is similar to mine, but the answer is not sufficiently ...
0
votes
1answer
13 views

Low leves of probability observed after modelling.Is it right to scale the probability

I have done modelling on imbalanced class , without any sampling methods. Event rate is around 0.1 ,After modelling I am getting probalities more at the lower side close to zero.I have tried differnt ...
1
vote
1answer
20 views

Can I use SVC() as a base_estimtor for ensemble methods?

I am currently testing out a few different ensemble methods on my dataset. I've heard that you can also use support vector machines as base learners in boosting and bagging methods but I am not sure ...
0
votes
0answers
13 views

Can bagging ensemble consist of heterogeneous base models?

Bagging or bootstrap aggregation seems to make sense for time series forecasting using an ensemble because bagging randomizes subsets of the data with replacement. However, I've only seen bagging used ...
0
votes
0answers
8 views

Creating a sub-model from pre-trained model

I have a pre-trained model having the following architecture: ...
0
votes
0answers
4 views

Domain specific language to describe ensemble model

I'm looking for some tool/library/widely used approach to describe Hierarchical Model structure like ensemble: It's absolutely straightforward how to do it with simple ensemble like It can be ...
0
votes
1answer
38 views

Can you combine two xgboost models into one?

If you have built two different xgbost models, with say 100 trees each, is it possible to combine into an xgboost model with 200 trees?
0
votes
1answer
72 views

What is the selection criteria to choose between XGBoost and Random Forest

I am trying to understand - when would someone choose Random Forest over XGBoost and vice versa. All the articles out there highlights on the differences between both. I understand them. But when ...
1
vote
0answers
207 views

ValueError: “The estimator should be a classifier”

I am adapting sklearn-extension ELMClassifier to be accepted as base_estimator to both VotingClassifier and AdaboostClassifier. ...
0
votes
0answers
13 views

Stacking using layer-1 models predictions on test set

I am new to Data Science and have been studying the methods of stacking to find out if it can meet the following fact, but I did not find or understand evidence that it can or cannot work. Let's ...
0
votes
0answers
13 views

Model ensemble - using model associated with median (instead of mean) for calculation purposes (Explainability)

I have seen many model ensemble litterature. Most, if not all of it, consider averaging models. I was considering using the median instead of the mean. In general I would consider this a good ...
1
vote
1answer
24 views

What should I use as training data for base (level 1) classifiers in ensembling?

Can I just take all training data that I have, train the base models on them and then take their results and use them for training level 2 model? Is this a good practice, or should it be done ...
1
vote
1answer
33 views

How many models shall be used in ensemble modelling?

I wish to make an ensemble of deep models to solve a classification problem. I want to know how many models shall be used to create that ensemble to ensure unbiased results. I have head 30+ models ...
0
votes
1answer
88 views

What's wrong with RF/SVM with word embedding (GloVe)?

I searched many times in google for examples on word embedding (specifically GloVe) with Random forest and I couldn't find any single example. For GloVe, it was all either LSTM or CNN. Maybe there's ...
2
votes
1answer
15 views

is it possible to use the train result as another feature and retrain?

is it possible to use the train result as another feature and retrain? for example I make prediction with classification and add this result to the table and train xgboost?
0
votes
0answers
19 views

Ensemble two models

I have regression task and I am predicting here with linear regression and randomforest models. Need some hints or code example how to ensemble them (averaging already done). Here are my model ...
2
votes
2answers
93 views

Hyperparameter optimization, ensembling instead of selecting with CV criteria

While burning CPUs performing a CV selection on a thin grid put on some hyperparameter space. I am using the `scikit-learn' API, for which the end result is a single point on the hyperparameter space, ...
1
vote
0answers
12 views

How to Predict Employee count of businesses using Keras classifiers

I am trying to predict the amount of employees a business has based on a set of input variables. I am using things like the business's age, transaction details, geographic location, business structure ...
2
votes
0answers
17 views

How to estimate the marginal distribution of a class with respect to one predictor in a classification task?

I have a dataset with a binary dependent variable $y \in \{0,1\}$ and a set of predictors $x1,x2,..,t$. Here, $t$ is the time in minutes (in 24 hrs, that is $t \in (0,1440)$). I want to estimate the ...
1
vote
1answer
15 views

When should we start using stacking of models?

I am solving a Kaggle contest and my single model has reached score of 0.121, I'd like to know when to start using ensembling/stacking to improve the score. I used lasso and xgboost and there ...
4
votes
2answers
375 views

SHAP value can explain right?

I face a problem with using SHAP value to interpret the Tree-based model. (https://github.com/slundberg/shapsd) First, I have input around 30 features and I have 2 features that have high positive ...
0
votes
0answers
17 views

Combining several Multi-Output-Models into a single Multi-Output-Model

I'm trying to create a k-Nearest-Neighbor based model of 76-dimensional input data $I$ and 44-dimensional output data $O$. Through domain knowledge I know that only certain input dimensions are ...
2
votes
0answers
21 views

Is there a universal convergence rate when stacking models/experts?

It's fairly common to see people stacking different models when chasing marginal gains in contexts such as Kaggle competitions or the Netflix challenge. I would like to know about the mathematics ...
0
votes
1answer
37 views

Deep model ensemble giving different results

I am making an ensemble of deep models for solving a classification problem. The initial weights follow the default distribution of keras layers. Each time I run ...
2
votes
2answers
232 views

Random forest vs majority voting

I'm using spark with scala to implement majority voting of decision trees and random forest (both are configured in the same way - same depth, the same amount of base classifiers etc.). Dataset is ...
1
vote
1answer
100 views

Combining multiple neural networks with different activation functions

I have 3 neural networks where each has as a different activation function: Sigmoid, Tanh and Softmax. I am planning to average their final predictions, but as we know the functions doesn't have the ...
0
votes
2answers
39 views

Drawing validation set from test set

I am building a 3 neural network models on dataset that is already separated to train and test sets. From my analysis, I found that this dataset has values on test set which don't exist in the train ...
1
vote
2answers
36 views

Ensemble Techniques - Bagging | Subset size

I do have a question on ensemble techniques Baggging/Boosting. - What would be the subset size for Bagging?
0
votes
1answer
22 views

Ensemble Techniques - Boosting

I understand boosting is a sequential learning technique and it use the prediction from previous model as a dataset for new model ,after adding weight to the misclassified data points. The point ...
1
vote
1answer
676 views

Training an ensemble of small neural networks efficiently in TensorFlow 2

I have a bunch of small neural networks (say, 5 to 50 feed-forward neural networks with only two hidden layers with 10-100 neurons each), which differ only in the weight initialization. I want to ...
2
votes
2answers
91 views

Gradient boosting how can accuracy increase when we lower the depth of tree?

What I don't understand about gradient boost is, doesn't lowering height of the tree means we use fewer features in our model? From my model I get the highest accuracy when the depth is one. Meaning ...
1
vote
0answers
9 views

Named Entity Recognition for Sesotho sa Leboa (Sepedi) one of the South African Official Language

I'm looking for model to used to develop NER for Sepedi?
0
votes
1answer
60 views

How to optimize hyperparameters in stacked model?

I was wondering whether somebody could explain how to optimize hyperparameters for the base learners and meta algorithm when stacking? In many tutorials they seem to be plucked out of thin air! ...
2
votes
1answer
73 views

How are Decision Trees averaged in Random Forest?

We all know that a Random Forest is an ensemble of Decision Trees, whose results are averaged. Every source I find simply talks about "averaging trees", but how does this "averaging of trees" happens?...
3
votes
1answer
94 views

Weighted Linear Combination of Classifiers

I am trying to build an ensemble of classifiers whereby I want my algorithm to learn a set of weights such that it can weight the outputs of different classifiers for a set of data points. I am ...
1
vote
0answers
12 views

Ensemble learning for multiple hypothesis classes

Just to confirm if the following description falls in the category of ensemble learning. Suppose given a training set $D=\{(X,Y)\}$ we are asked to train a regressor. But now the way we do it is to ...