Questions tagged [ensemble-modeling]

In machine learning, ensemble methods combine multiple algorithms to make a prediction. Bagging, boosting, and stacking, are some examples.

Filter by
Sorted by
Tagged with
1
vote
1answer
19 views

Combining multiple neural networks with different activation functions

I have 3 neural networks where each has as a different activation function: Sigmoid, Tanh and Softmax. I am planning to average their final predictions, but as we know the functions doesn't have the ...
0
votes
1answer
16 views

Drawing validation set from test set

I am building a 3 neural network models on dataset that is already separated to train and test sets. From my analysis, I found that this dataset has values on test set which don't exist in the train ...
1
vote
1answer
12 views

Ensemble Techniques - Bagging | Subset size

I do have a question on ensemble techniques Baggging/Boosting. - What would be the subset size for Bagging?
0
votes
1answer
19 views

Ensemble Techniques - Boosting

I understand boosting is a sequential learning technique and it use the prediction from previous model as a dataset for new model ,after adding weight to the misclassified data points. The point ...
1
vote
0answers
23 views

How to stack classifiers optimized for different score functions?

I have binary classification task (class0 vs class1) and I would like to create a stacked model out of classifiers which are individually optimized for different scorings. For example, let's say Clf_A ...
1
vote
1answer
100 views

Training an ensemble of small neural networks efficiently in TensorFlow 2

I have a bunch of small neural networks (say, 5 to 50 feed-forward neural networks with only two hidden layers with 10-100 neurons each), which differ only in the weight initialization. I want to ...
2
votes
2answers
42 views

Gradient boosting how can accuracy increase when we lower the depth of tree?

What I don't understand about gradient boost is, doesn't lowering height of the tree means we use fewer features in our model? From my model I get the highest accuracy when the depth is one. Meaning ...
1
vote
0answers
9 views

Named Entity Recognition for Sesotho sa Leboa (Sepedi) one of the South African Official Language

I'm looking for model to used to develop NER for Sepedi?
0
votes
1answer
33 views

How to optimize hyperparameters in stacked model?

I was wondering whether somebody could explain how to optimize hyperparameters for the base learners and meta algorithm when stacking? In many tutorials they seem to be plucked out of thin air! ...
0
votes
0answers
28 views

How to add weight to a variable to a ensemble method

I am building an ensemble method and would like to add weight to one particular variable (x3 in this case). How can I do this? Here is my code: ...
2
votes
1answer
48 views

How are Decision Trees averaged in Random Forest?

We all know that a Random Forest is an ensemble of Decision Trees, whose results are averaged. Every source I find simply talks about "averaging trees", but how does this "averaging of trees" happens?...
0
votes
0answers
10 views

Manual setting of target variable based on features' minimum values: f1 score = 1

I am building a classifier for user engagement in my website. Basically, since there are no "proxy" for engagement, i.e. there is no pre-defined target variable, I came up with minimum thresholds ...
3
votes
1answer
39 views

Weighted Linear Combination of Classifiers

I am trying to build an ensemble of classifiers whereby I want my algorithm to learn a set of weights such that it can weight the outputs of different classifiers for a set of data points. I am ...
0
votes
0answers
32 views

How to deal with small amount of labeled samples?

I'm trying to develop skill to deal with very small amount of labeled samples (250 labeled/20000 total, 200 features) practicing on Kaggle "Don't Overfit" dataset (Traget_Practice have provided all 20,...
1
vote
0answers
10 views

Ensemble learning for multiple hypothesis classes

Just to confirm if the following description falls in the category of ensemble learning. Suppose given a training set $D=\{(X,Y)\}$ we are asked to train a regressor. But now the way we do it is to ...
1
vote
1answer
42 views

Understanding why ensembling only improves marginally

I have two models, A and B, trained on Imagenet. Their accuracies on Imagenet validation set are 35.6% and 28.64% respectively, while the accuracy of their ensemble (averaging their scores) is 35.68%. ...
1
vote
0answers
11 views

Would it make technical sense to model its two known subelements separately (CLV example)?

Let's assume Customer Lifetime Value (CLV) is defined as average basket x frequency. Option A is to build model predicting CLV ...
1
vote
1answer
105 views

checking model stability - Performance for different class

I tried to do multi-class classification problem. The goal is to predict whether the match will be won by HomeTeam, AwayTeam or Draw. I did feature engineering from the attributes and finally came up ...
1
vote
0answers
50 views

Recommendations for Neural Network Stacking Project

I embarked in a project to use a stacking ensemble of neural networks to perform a binary classification prediction. I have the ensemble running and making decent accuracy estimates (90% ca.). ...
1
vote
1answer
1k views

Is there any implementation of Extended Isolation Forest algorithm in R/Python?

I am using isofor package for regular Isolation Forest but I came by an article about Extended Isolation Forest and need your advise which package has this ...
1
vote
0answers
150 views

Ensemble Method using XGBoost and RotationForest python

How can I create an ensemble model using XGBoost and Rotation Forest in Python?
1
vote
1answer
97 views

Voting classifier using grid search for Time Series

I have three models: Arima Auto ARIMA Double Exponential Smoothing I would like to apply an ensemble method - a voting method and allow the classifier to learn weights for these three models. I ...
4
votes
1answer
38 views

Building Stacking machine learning model using three base classifiers

I did a stacking using three base classifiers RF, NB, KN N and metamodel random forest or SVM using sklearn library But which is strange each time i change the metamodel i got the same results. Is it ...
2
votes
0answers
37 views

Is linear regression on the trees of XGBoost (rather than taking their mean) useful/popular?

Given training data $(\underline{x}_1, y_1),...,(\underline{x_N}, y_N)$, one can choose a variety of ensemble method for trees. These algorithms output a set of trees $T_1, ..., T_n$, and then the ...
1
vote
1answer
205 views

Difference between bagging and boosting

Can anyone explain me the basic difference between bagging and boosting and which technique can be used in which scenario?
3
votes
1answer
5k views

Adaboost vs Gradient Boosting

How AdaBoost is different than Gradient Boosting algorithm since both of them works on Boosting technique? I could not figure out actual difference between these both algorithms from theory point of ...
1
vote
1answer
223 views

best activation function for ensemble?

i have created some logistic regression model (different preprocessing) with softmax function. and i mix all model with an ensemble with a hierarchical method. so the output of all model (base) will ...
2
votes
2answers
47 views

What methods can be used to detect duplicacy in image dataset?

I want to remove duplicate images from a dataset of 50Million images. What is the best method to detect all the duplicates? Do you think one shot learning is good for this?
1
vote
1answer
168 views

Why Extra-trees should only be used within ensemble methods?

I was reading scikit-learn documentation for Extremely Randomized Trees and I found this warning: Warning: Extra-trees should only be used within ensemble methods. Why is that?
1
vote
1answer
54 views

Combining outputs of ridge regression models?

I am facing an issue where I have 7 sets of different variables/columns/predictors. I am trying to predict same target variable and I want to observe the importance/effect of all the sets according ...
1
vote
0answers
26 views

Why should each layer's child network output be close to parent network's output for variance regularizer?

I am reading up on PEA (Pseudo ensemble agreement) regularizer. specificaly in the neural networks domain. It introduces the concept of perturbing the model a little and forcing the model to make ...
1
vote
0answers
326 views

Combining image and scalar inputs into a neural network

I'm looking at the best way of combining CNN with image input and a scalar value. I know that one of the ways is to concatenate flatten layer with this scalar value. But flatten layer consist for ...
1
vote
0answers
65 views

heterogeneous input features for estimators in sklearn.ensemble.VotingClassifier

I would like to do ensemble on my model. Two of them are SVM and XGBoosting. SVM could not tolerate null value and XGB can do it. So I have different features for each of them. but when ...
0
votes
2answers
1k views

Categorical data for sklearns Isolation Forrest

I'm trying to do anomaly detection with Isolation Forests (IF) in sklearn. Except for the fact that it is a great method of anomaly detection, I also want to use it because about half of my features ...
3
votes
2answers
273 views

What is the meaning of the term “pipeline” within data science?

Note: this question was asked and removed just before I posted my answer below, so am repeating the general idea here People often refer to pipelines when talking about models, data and even layers ...
0
votes
2answers
80 views

Why does prediction by a consensus of classifier work better than prediction by a single classifiers?

I have seen that consensus of classifiers (taking say 5 separate classifiers) and obtaining the final labeling of the unknown sample based on the voting method (whichever class gets the predicted the ...
0
votes
2answers
65 views

Ensemble models - neural network input both original data and predictions of other models?

From my understanding in order to improve accuracy with ensemble models you need a wide range of independent ensemble methods. I was wondering whether using the ouput of a random forest model as one ...
2
votes
0answers
65 views

Prevent overffitting in model stacking with training on the same target

I'm trying to solve Quora Question Pairs with model stacking. My first layers are: CNN trained to predict the same target as whole model should "Magic features" like question frequency in whole ...
1
vote
1answer
64 views

How to construct a self learning process for an ensemble model?

I have two different (regression) models spitting out predictions on a daily basis for the same dependent variable. My intention is to assign weights to those two predictions and calculate a weighted ...
4
votes
1answer
562 views

Geometric and harmonic means in ensembling methods

When using ensembling methods for regression, a common approach is to average (using the arithmetic mean) the outputs of the weak learners in order to obtain the output of the ensemble. Is there a ...
0
votes
2answers
5k views

How to avoid memory error with Pandas pd.read_csv method call with GridSearchCV usage for DecisionTreeRegressor model?

I have been implementing a DecisionTreeRegressor model in Anaconda environment with a data set sourced from a 20 million row, 12-dimensional CSV file. I could get the chunks off of the data set with ...
6
votes
3answers
127 views

What are the individual models within a machine learning ensemble called?

I am aware that an ensemble machine learning model is a stack of two or more machine learning models. Is there a word to refer to those individual models that go into the ensemble model? (i.e. a ...
1
vote
2answers
92 views

Questions on ensemble technique in machine learning

I am studying the ensemble machine learning and when I read some articles online, I encountered 2 questions. 1. In this article, it mentions Instead, model 2 may have a better overall performance ...
1
vote
1answer
110 views

Multi-dimensional regression with ensemble models?

I'm curious to know whether boosting, random forests or other types of ensemble models can perform multi-dimensional regression. To be precise: That means multiple outputs (multi-dimensional labels) ...
1
vote
0answers
40 views

AdaBoost - Ensemble model perform poor than best weak classifier

Can Adaboost's ensemble classifier perform worse than the best of the weak learners considered? If so when in what case of weak learner the ensemble learning does not perform better?
1
vote
1answer
24 views

How to approach model reporting task

I have been tasked to report on an ensemble model that was created in h2o which includes several model subtypes such as Random Forest, GBM, linear models etc. The end goal is to predict churn rates ...
0
votes
2answers
168 views

Retrieve user features in real time from UserId for prediction

Let's say I'm building an app like Uber and I want to predict the user's most likely destination based on the user's past history, current latitude/longitude, and time/date. Here is the proposed ...
1
vote
1answer
140 views

testing new data in model

I have ensembled 3 algorithms as below, ...
2
votes
2answers
110 views

Combining Different Models

I have N models, each of them are used to predict on a set of data. I am currently combining their predictions by averaging across rows. Need suggestions on combining their predictions. End goal is to ...
1
vote
1answer
25 views

Re: Missing Value

I have a large dataset that has session length records per user basis. And I am trying to predict the purchase behaviour based on the session length. But this data has multiple zero in the session ...