Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

The tag has no usage guidance.

1
vote
0answers
24 views

Why should each layer's child network output be close to parent network's output for variance regularizer?

I am reading up on PEA (Pseudo ensemble agreement) regularizer. specificaly in the neural networks domain. It introduces the concept of perturbing the model a little and forcing the model to make ...
1
vote
0answers
19 views

Combining image and scalar inputs into a neural network

I'm looking at the best way of combining CNN with image input and a scalar value. I know that one of the ways is to concatenate flatten layer with this scalar value. But flatten layer consist for ...
1
vote
0answers
5 views

heterogeneous input features for estimators in sklearn.ensemble.VotingClassifier

I would like to do ensemble on my model. Two of them are SVM and XGBoosting. SVM could not tolerate null value and XGB can do it. So I have different features for each of them. but when ...
0
votes
1answer
21 views

Categorical data for sklearns Isolation Forrest

I'm trying to do anomaly detection with Isolation Forests (IF) in sklearn. Except for the fact that it is a great method of anomaly detection, I also want to use it because about half of my features ...
0
votes
0answers
5 views

How to optimize stacking?

I'm wondering if there is a way to find the optimal weights when stacking multiple models? For example if I have five models which perform similarly, how do I find if I should discard some altogether, ...
0
votes
1answer
28 views

What is the meaning of the term “pipeline” within data science?

Note: this question was asked and removed just before I posted my answer below, so am repeating the general idea here People often refer to pipelines when talking about models, data and even layers ...
0
votes
2answers
29 views

Why does prediction by a consensus of classifier work better than prediction by a single classifiers?

I have seen that consensus of classifiers (taking say 5 separate classifiers) and obtaining the final labeling of the unknown sample based on the voting method (whichever class gets the predicted the ...
0
votes
0answers
18 views

Selecting the best combination of machine learning models for voting

I am thinking about using Sklearn's VotingClassifer for a dataset. I have heard about people winning in machine learning competitions (like those from Kaggle) by correctly utilizing voting/stacking. ...
0
votes
1answer
16 views

Ensemble models - neural network input both original data and predictions of other models?

From my understanding in order to improve accuracy with ensemble models you need a wide range of independent ensemble methods. I was wondering whether using the ouput of a random forest model as one ...
2
votes
0answers
45 views

Prevent overffitting in model stacking with training on the same target

I'm trying to solve Quora Question Pairs with model stacking. My first layers are: CNN trained to predict the same target as whole model should "Magic features" like question frequency in whole ...
1
vote
1answer
37 views

How to construct a self learning process for an ensemble model?

I have two different (regression) models spitting out predictions on a daily basis for the same dependent variable. My intention is to assign weights to those two predictions and calculate a weighted ...
2
votes
1answer
84 views

Geometric and harmonic means in ensembling methods

When using ensembling methods for regression, a common approach is to average (using the arithmetic mean) the outputs of the weak learners in order to obtain the output of the ensemble. Is there a ...
0
votes
2answers
973 views

How to avoid memory error with Pandas pd.read_csv method call with GridSearchCV usage for DecisionTreeRegressor model?

I have been implementing a DecisionTreeRegressor model in Anaconda environment with a data set sourced from a 20 million row, 12-dimensional CSV file. I could get the chunks off of the data set with ...
6
votes
3answers
90 views

What are the individual models within a machine learning ensemble called?

I am aware that an ensemble machine learning model is a stack of two or more machine learning models. Is there a word to refer to those individual models that go into the ensemble model? (i.e. a ...
0
votes
2answers
46 views

Questions on ensemble technique in machine learning

I am studying the ensemble machine learning and when I read some articles online, I encountered 2 questions. 1. In this article, it mentions Instead, model 2 may have a better overall performance ...
0
votes
0answers
35 views

Ensemble Classifier (brew package) - KeyError: '[290 109 240 11 524] not in index'

I am using Ensemble Classifiers from the package brew 0.1.4 and was trying to create dynamic selection classifier from the following example. So my code snippet is like this: ...
0
votes
1answer
36 views

Multi-dimensional regression with ensemble models?

I'm curious to know whether boosting, random forests or other types of ensemble models can perform multi-dimensional regression. To be precise: That means multiple outputs (multi-dimensional labels) ...
1
vote
0answers
23 views

AdaBoost - Ensemble model perform poor than best weak classifier

Can Adaboost's ensemble classifier perform worse than the best of the weak learners considered? If so when in what case of weak learner the ensemble learning does not perform better?
0
votes
1answer
17 views

How to approach model reporting task

I have been tasked to report on an ensemble model that was created in h2o which includes several model subtypes such as Random Forest, GBM, linear models etc. The end goal is to predict churn rates ...
0
votes
1answer
49 views

Retrieve user features in real time from UserId for prediction

Let's say I'm building an app like Uber and I want to predict the user's most likely destination based on the user's past history, current latitude/longitude, and time/date. Here is the proposed ...
1
vote
1answer
41 views

testing new data in model

I have ensembled 3 algorithms as below, ...
1
vote
2answers
49 views

Combining Different Models

I have N models, each of them are used to predict on a set of data. I am currently combining their predictions by averaging across rows. Need suggestions on combining their predictions. End goal is to ...
1
vote
1answer
22 views

Re: Missing Value

I have a large dataset that has session length records per user basis. And I am trying to predict the purchase behaviour based on the session length. But this data has multiple zero in the session ...
3
votes
2answers
232 views

Taking average of multiple neural networks?

I'm fitting a neural network using a very small data set, so try splitting the data into training and validation sets. (there is a separate test set) If I split training/validation randomly multiple ...
2
votes
2answers
87 views

How to create an ensemble that gives precedence to a specific classifier

Suppose that in a binary classification task, I have separate classifiers A, B, and C. If I ...
0
votes
0answers
23 views

How to Compare resampling methods (Dolan-More Curve)

I read about comparing data resampling methods in this publication: https://arxiv.org/abs/1707.03905 In it, they used the Dolan-More Curve. I have been trying to read more about it but can't find ...
0
votes
0answers
36 views

Can I automate stacking?

I have learned that stacking different methods works well for machine learning problems? Do we have a clear text (any book/article/blog) that can state which methods to stack were in the layers (base ...
0
votes
2answers
65 views

Ensemble Model vs Normal model

If I get 95+ % accuracy in normal models, should I still consider Ensemble models? Why should I choose Ensemble models over normal models?
-1
votes
1answer
69 views

Ensemble Probabilities of the different models

I have a Multiclass Dataset and I am getting probabilities of classes from RandomForest. However, I want to divide the dataset for each class as examples of either the case belongs to that class or ...
-1
votes
1answer
36 views

How to average classifiers with AUC metric?

I am modeling a binary classification and my loss function is the gini function (normalized area under the curve). Here's my implementation: Split the data with k-folds Train k classifiers Now I ...
1
vote
2answers
63 views

Learning Algorithm that decide which model gives better results for each testing instance

Is their any existing Ensemble technique which uses subset of training data to predict which algorithm is better for predicting each instance of testing data? Let's say we have N sized training set ...
1
vote
0answers
345 views
1
vote
1answer
728 views

Sklearn Aggregating Multiple Fitted Models Into A Single Model? (binary classification)

My problem context: dataset too big to fit into memory. binary classification [0,1] 30 csv files in a directory with exactly 30,000 samples (rows) each file contains 15,000 ...
0
votes
0answers
27 views

Systematic combination of already accurate classifiers

For a document classification task, I am equipped with at least two already good performing classifiers in terms of accuracy (ca. 80% for a classification task involving 7 classes). Since both ...
0
votes
0answers
33 views

Tuning Adaboost to prevent equal confidence levels

I am working on a financial data set and I am using the boosting ensemble method Adaboost (tree) to classify an upward or downward return for a month. I use several technical indicators (normalized ...
0
votes
1answer
717 views

How can I do tree_method ='exact' in XGBoost classifier?

I am doing XGBoost classification on a huge data set and its showing: ...
2
votes
2answers
175 views

How to measure the correlation of different algorithms

In stacked generalization, several algorithms are trained on the training set (i.e. at layer 1) and their predictions are then stacked using a layer 2 model. In many documentations, it is said that it ...
0
votes
1answer
30 views

What kind of algorithms can be used as a stacker in stacked generalization

In stacked generalization, several algorithms (I use some random trees, booster trees, ...) are first trained and used to make the predictions, which are used as input for another algorithm. However, ...
4
votes
1answer
85 views

Why some people add results from PCA and other dimensional reductions techniques as features

I often see in a well-known datascience competition platform, that a lot of people apply some dimension reductions techniques, but instead of using it to reduce the number of features (complexity) of ...
5
votes
1answer
92 views

How predictions of level 1 models become training set of a new model in stacked generalization.

In stacked generalization, if I understood well, we divide the training set into train/test set. We use train set to train M models, and make predictions on test set. Then we use the predictions as ...
1
vote
2answers
64 views

How to use an existing model as in input into a new model

We have a click-model which is currently being used for search ranking in production, and I want to create a new model which takes the old model click probability as one input and adds some other ...
0
votes
2answers
171 views

Ensemble learning [closed]

I am currently working to build a mathematical model to predict the stock market. I learned that the best way to do such thing is no longer to make one big best model, but rather to gather several ...
3
votes
2answers
667 views

Improving classifier performances in R for imbalanced dataset

I have used an "adabag"(boosting + bagging) model on an imbalanced dataset (6% positive), I have tried to maximized the sensitivity while keeping the accuracy above 70% and the best results I got were:...
0
votes
1answer
1k views

Ensemble modelling using model's probabilities

In a classification project, on the training sets, I ran a selection of classifiers. These give me about 20-30% accuracy at best. For each sample, I generate probabilities of each class. I want to ...
1
vote
2answers
458 views

Is there an R package for Locally Interpretable Model Agnostic Explanations?

One of the researchers, Marco Ribeiro, who developed this method of explaining how black box models make their decisions has developed a Python implementation of the algorithm available through Github,...
2
votes
1answer
38 views

How to do boosting in model-ensembling?

Boosting is a sequential technique in which, the first algorithm is trained on the entire dataset and the subsequent algorithms are built by fitting the residuals of the first algorithm, thus giving ...
5
votes
4answers
189 views

Does ensemble (bagging, boosting, stacking, etc) always at least increase performance?

Ensembling is getting more and more popular. I understand that there are in general three big fields of ensembling, bagging, boosting and stacking. My question is that does the ensembling always at ...
25
votes
4answers
25k views

GBM vs XGBOOST? Key differences?

I am trying to understand the key difference between GBM and XGBOOST. I tried to google it but could not find any good answer explaining the difference between the two algos and why does xgboost ...
2
votes
1answer
74 views

Decision Tree Ensembling

In order to ensemble a decision tree, let me explain the specific situation. I have split the dataset in 5 sections, per protocol e.g. TCP, HTTP etc. Now I've trained a decision tree for each one and ...
1
vote
1answer
532 views

How to use ensemble of models in FM or FFM?

I am using Factorization Machines ( libfm) and also the Field Aware Factorization Machines (libffm) for a kaggle competition. I am currently using the single models of each respectively for ...