Questions tagged [ensemble-modeling]

In machine learning, ensemble methods combine multiple algorithms to make a prediction. Bagging, boosting, and stacking, are some examples.

44 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
0answers
2k views

How to tune weights in Voting Classifier (Sklearn)

I am trying to do the following: ...
3
votes
3answers
57 views

Understanding Weighted learning in Ensemble Classifiers

I'm currently studying Boosting techniques in Machine Learning and I happened to understand that in Algorithms like Adaboost, each of the training samples is given a weight depending on whether it was ...
3
votes
0answers
51 views

Is linear regression on the trees of XGBoost (rather than taking their mean) useful/popular?

Given training data $(\underline{x}_1, y_1),...,(\underline{x_N}, y_N)$, one can choose a variety of ensemble method for trees. These algorithms output a set of trees $T_1, ..., T_n$, and then the ...
2
votes
0answers
45 views

How to improve model performace when model shows a systemic pattern in residues

I'm working on a regression model using Boosting algorithms (CatBoost, XGBoost, and LightGBM). All models give similar accuracy of 0.2 RMSE (Target varies from 0 to 1). I obtained the following plots ...
2
votes
0answers
51 views

How to estimate the marginal distribution of a class with respect to one predictor in a classification task?

I have a dataset with a binary dependent variable $y \in \{0,1\}$ and a set of predictors $x1,x2,..,t$. Here, $t$ is the time in minutes (in 24 hrs, that is $t \in (0,1440)$). I want to estimate the ...
2
votes
0answers
23 views

Is there a universal convergence rate when stacking models/experts?

It's fairly common to see people stacking different models when chasing marginal gains in contexts such as Kaggle competitions or the Netflix challenge. I would like to know about the mathematics ...
2
votes
1answer
1k views

Training an ensemble of small neural networks efficiently in TensorFlow 2

I have a bunch of small neural networks (say, 5 to 50 feed-forward neural networks with only two hidden layers with 10-100 neurons each), which differ only in the weight initialization. I want to ...
2
votes
0answers
78 views

Averaging CNN perform worse than boosting

I'm trying to solve Quora Question Pairs with model stacking. My first layers are: CNN trained to predict the same target as whole model should "Magic features" like question frequency in ...
1
vote
1answer
29 views

Making an ensemble model for high F1 score

I presently have 2 algorithms that have a numerical output. Using a threshold of 0.9, I get the classification output. Let's say they are: P (high precision, low recall) R (high recall, low precision)...
1
vote
0answers
9 views

Large general-purpose model vs ensemble of many smaller models

I am reading this paper - https://arxiv.org/abs/1503.02531v1 - devoted to knowledge distillation in neural networks. One interesting approach is mentioned in this paper in sections ...
1
vote
0answers
41 views

How does stacking help Bias and Variance?

How does stacking help in terms of bias and variance? I have a hunch that stacking can help reduce bias but i am not sure, could someone refer to a paper?
1
vote
0answers
91 views

Stacking - Appropriate base and meta models

When implementing stacking for model building and prediction (For example using sklearn's StackingRegressor function) what is the appropriate choice of models for the base models and final meta model?...
1
vote
2answers
19 views

Selecting models for ensemble from large group of models with high uncertaintly

I'm in a situation where many models have been created, and I have their cross-validation performances as well as performance on test data. I need to select models for inclusion in a simple bagging ...
1
vote
0answers
15 views

General practices for building an incremental learning model which never forgets?

I'm new to datascience and appreciate your sage advice! I need to build an incremental learning model, and I know there's a lot that goes into something like that, but I'd like to highlight the most ...
1
vote
0answers
20 views

Decision Boundary with random new observations vs observations from test set

I'm trying to plot decision boundary for Decision Tree classifier. Classifier is trained on training set, and decision boundary (contour) using random new observations and observations from test set ...
1
vote
0answers
78 views

Can bagging ensemble consist of heterogeneous base models?

Bagging or bootstrap aggregation seems to make sense for time series forecasting using an ensemble because bagging randomizes subsets of the data with replacement. However, I've only seen bagging used ...
1
vote
0answers
30 views

Creating a sub-model from pre-trained model

I have a pre-trained model having the following architecture: ...
1
vote
0answers
11 views

Domain specific language to describe ensemble model

I'm looking for some tool/library/widely used approach to describe Hierarchical Model structure like ensemble: It's absolutely straightforward how to do it with simple ensemble like It can be ...
1
vote
0answers
13 views

How to Predict Employee count of businesses using Keras classifiers

I am trying to predict the amount of employees a business has based on a set of input variables. I am using things like the business's age, transaction details, geographic location, business structure ...
1
vote
0answers
14 views

Ensemble learning for multiple hypothesis classes

Just to confirm if the following description falls in the category of ensemble learning. Suppose given a training set $D=\{(X,Y)\}$ we are asked to train a regressor. But now the way we do it is to ...
1
vote
0answers
20 views

Would it make technical sense to model its two known subelements separately (CLV example)?

Let's assume Customer Lifetime Value (CLV) is defined as average basket x frequency. Option A is to build model predicting CLV ...
1
vote
1answer
513 views

checking model stability - Performance for different class

I tried to do multi-class classification problem. The goal is to predict whether the match will be won by HomeTeam, AwayTeam or Draw. I did feature engineering from the attributes and finally came up ...
1
vote
0answers
204 views

Ensemble Method using XGBoost and RotationForest python

How can I create an ensemble model using XGBoost and Rotation Forest in Python?
1
vote
1answer
245 views

Voting classifier using grid search for Time Series

I have three models: Arima Auto ARIMA Double Exponential Smoothing I would like to apply an ensemble method - a voting method and allow the classifier to learn weights for these three models. I ...
1
vote
0answers
552 views

Combining image and scalar inputs into a neural network

I'm looking at the best way of combining CNN with image input and a scalar value. I know that one of the ways is to concatenate flatten layer with this scalar value. But flatten layer consist for ...
1
vote
0answers
87 views

heterogeneous input features for estimators in sklearn.ensemble.VotingClassifier

I would like to do ensemble on my model. Two of them are SVM and XGBoosting. SVM could not tolerate null value and XGB can do it. So I have different features for each of them. but when ...
1
vote
1answer
9k views

How to avoid memory error with Pandas pd.read_csv method call with GridSearchCV usage for DecisionTreeRegressor model?

I have been implementing a DecisionTreeRegressor model in Anaconda environment with a data set sourced from a 20 million row, 12-dimensional CSV file. I could get the chunks off of the data set with ...
1
vote
0answers
59 views

AdaBoost - Ensemble model perform poor than best weak classifier

Can Adaboost's ensemble classifier perform worse than the best of the weak learners considered? If so when in what case of weak learner the ensemble learning does not perform better?
1
vote
3answers
225 views

Retrieve user features in real time from UserId for prediction

Let's say I'm building an app like Uber and I want to predict the user's most likely destination based on the user's past history, current latitude/longitude, and time/date. Here is the proposed ...
1
vote
0answers
500 views

Compare probability estimates of two classifiers

Consider two binary classifiers A and B. Suppose that both A and B are predicting the same target, but that A is trained on a subset of the data for which a different set of features is available than ...
0
votes
0answers
23 views

How does XGBoost perform in Parallel

So what I know about boosting technique, Like we train the data and update the weights of falsely predicted values or try to minimize the loss in the next model. So basically, it's the sequential ...
0
votes
0answers
10 views

How can I use Ensemble learning of two models with different features as an input?

I have a fake news detection problem and it predicts the binary labels "1"&"0" by vectorizing the 'tweet' column, I use three different models for detection but I want to use ...
0
votes
0answers
5 views

Ensemble/combining models weighted by number of observations?

Across a few different projects, I have hit a problem where I have two (or more) models: General-Purpose Model: A model which is based on a large amount of data not specifically relevant to my ...
0
votes
0answers
19 views

Stacking Classifier Error

I am using the Stacking classifier and getting following Error CODE ...
0
votes
0answers
17 views

How to make an lstm ensemble with different input shapes using keras

This is what I got so far for making an lstm ensemble with one model input for each of the lstm models and for the ensemble model and it works perfectly. ...
0
votes
0answers
15 views

Hybrid Ensembling

I read a paper recently where researchers were trying to do classification using ensembling methods. I first read about the concept of BOOSTED-STACKING there.For those who dont know , I can give a ...
0
votes
0answers
17 views

How to interpret gradient descent in boosting ensembles?

I struggle to grasp the role of gradient based optimization in boosting ensembles. As far as I understand boosting means combining a bunch of estimators (of the same types, usually decision trees) ...
0
votes
0answers
44 views

How do I proceed with model selection?

Only data scientist in an organization and I could really use a sounding board here. In Phase One of a project I deployed four models and served their average as the prediction. I used Random Forest ...
0
votes
0answers
21 views
0
votes
1answer
27 views

Model with 2 datasets: combine time series data and statistics

I am new to data science modelling so apologies if using wrong terminology in advance. I have a standard time series dataset of historical prices which is used to train/test a simple Random Forest ...
0
votes
0answers
67 views

What is the best way to use Early Stopping in an ensemble (stacking) model?

I have a training and a test dataset. I would like to use the output of Model A in an ensemble model. However, I would like to use early stopping. Usually, I would create Model A for each K-fold (on ...
0
votes
1answer
922 views

XGBoost Log Loss different from GridSearchCV Log Loss

I have a classification problem where I am trying to predict if the data returns a 1 or 0. So your classic binary classification. I have my set of data that I have split into the dependent variables (...
0
votes
0answers
21 views

Combining several Multi-Output-Models into a single Multi-Output-Model

I'm trying to create a k-Nearest-Neighbor based model of 76-dimensional input data $I$ and 44-dimensional output data $O$. Through domain knowledge I know that only certain input dimensions are ...
0
votes
2answers
89 views

Drawing validation set from test set

I am building a 3 neural network models on dataset that is already separated to train and test sets. From my analysis, I found that this dataset has values on test set which don't exist in the train ...