We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [random-forest]

Random forest is a machine-learning classifier based on choosing random subsets of variables for each tree and using the most frequent tree output as the overall classification.

Filter by
Sorted by
Tagged with
2
votes
0answers
7 views

When does random forest feature importance fail?

I'm curious about the assumptions of random forest feature importance. In this paper, the author says that "We show that random forest variable importance measures are a sensible means for ...
0
votes
0answers
9 views

Random Forest proximity matrix with or without in bag samples

I'm running Random Forest classifications; and PCA plots based on the resulting proximity matrices are quite helpful to my analyses. However, I found a comment on the R ...
0
votes
1answer
21 views

Permutation feature importance vs. RandomForest feature importance

What is the difference between Permutation feature importance vs. RandomForest feature importance? What are the disadvantages vs. advantages of the two techniques?
0
votes
1answer
24 views

classifier predicts only one class [on hold]

I have a classification that has to predict three different classes: gcc,icc, clang. The prblem is that if I use a blind test set to do a submission, when I look athe the prediction I have on it I ...
2
votes
2answers
46 views

Incorrectly applying random forest model?

I am fairly new to random forest models (and data science in general), and was wondering if I am operationalizing the model I created correctly. Context: I am creating a random forest model to ...
0
votes
1answer
35 views

Decision tree and random forest over fitting

I am working on a real state data set to predict the price of buying a house in Dubai based on area, no.of bedrooms, number of baths and the town which the house is in. All variables are numerical ...
3
votes
1answer
49 views

The output of Model from Decision Tree and Random Forests are different?

I have been made a model using both Decision Tree and Random Forest. But, when I tried to test the model on the same DataFrame the output is different. How is this possible? The data file from my ...
1
vote
1answer
27 views

Finding out which values lead Random tree to a decision

I have a dataset of machines that produce plastic parts. A camera evaluates whether a plastic part was produced correctly or not (binary classification). I'm trying to figure out which factors ...
4
votes
2answers
121 views

Time series forecasting dilemma. Could feature engineering overcome time dependency?

I keep reading articles about time series forecasting. They all start from the same assumption: time series forecasting can't be treated as a regression/classification problem. It is time dependent, ...
1
vote
2answers
32 views

Majority voting in scikit-learn Random forest

My main concern is that i need to understand that how does the random forest do majority voting in scikit learn source code. I did not find that specific code in source code of RandomForest. if ...
0
votes
1answer
35 views

Normalize / Standardize in a Random Forest?

If I have a matrix of co-occurring words in conversations of different lengths, is it appropriate to standardize / normalize the data prior to training? My matrix is set up as follows: one row per ...
0
votes
0answers
10 views

Improve the results of imbalanced multi-classification multi-lables data

I have 10k rows of multi-classification (x1..x27,y), size of dataframe is: 28*10k and its ...
1
vote
1answer
19 views

Why does removal of some features improve the performance of random forests on some occasions?

I completed feature importance of a random forest model. I removed the bottom 4 features out of 17 features. The model performance actually improved. Shouldn't the performance degrade after removal of ...
0
votes
1answer
25 views

Explaining feature_importances_ in Scikit Learn RandomForestRegressor

For a project, I used the feature_importances_ attributes from the RandomForestRegressor. Everything works well but I don't know ...
2
votes
2answers
38 views

Random forest vs majority voting

I'm using spark with scala to implement majority voting of decision trees and random forest (both are configured in the same way - same depth, the same amount of base classifiers etc.). Dataset is ...
1
vote
0answers
25 views

Rescale prediction to correct dispersion due to correlation between response and residuals in Random Forrest Regression

I am using Random Forest Regression and I observe a strong positive correlation between the residuals ($\hat{u}$) and the response variable ($y$) which lead to a dispersion : predictions are ...
4
votes
2answers
72 views

Bagging vs Boosting, Bias vs Variance, Depth of trees

I understand the main principle of bagging and boosting for classification and regression trees. My doubts are about the optimization of the hyperparameters, especially the depth of the trees First ...
0
votes
2answers
49 views

Are validation sets necessary for Random Forest Classifier?

Is it necessary to have train, test and validation sets when using random forest classifier? I understand it is important with Neural Networks but I am not understanding the importance of it with RF. ...
2
votes
2answers
109 views

Can a decision tree learn to solve a xOR problem?

I have read online that decision trees can solve xOR type problems, as shown in images (xOR problem: 1) and (Possible solution as decision tree: 2). My question is how can a decision tree learn to ...
0
votes
0answers
17 views

IndexError: too many indices for array

Attempting to load a dataset and print out a prediction for each array, yet I keep getting an IndexError.... ...
1
vote
1answer
39 views

random forest classifier - impact of small n_estimator and repeated training

trying to have a better understanding of random forest algorithm here. With the same training and holdout datasets, I tried two things here: Set a small n_estimator (10), train on my training dataset ...
3
votes
3answers
88 views

Why did sampling boost the performance of my model?

I have an imbalanced dataset with 88 positive samples and 128575 negative samples. I was reluctant to over/undersample the data since it's a biological dataset and I didn't want to introduce synthetic ...
1
vote
1answer
41 views

how to use word embedding to do document classification etc?

I just start learning NLP technology, such as GPT, Bert, XLnet, word2vec, Glove etc. I try my best to read papers and check source code. But I still cannot understand very well. When we use word2vec ...
1
vote
0answers
13 views

Variable importance of Numerical features in Classification Model - Random Forest Classifier

I have few numeric features in my model. Out of 25 features, I have 7-8 numeric features in my model. One thing I observed is model gives more weightage to numerical feature compare to categorical ...
1
vote
0answers
42 views

Trying to beat random forest with xgboost

I have a small time series dataset of about 3000 samples and 5 features. With xgboost, my predictions seem biased (consistently overestimating the target). No matter how many estimators I throw at the ...
1
vote
1answer
29 views

Implicit feature selection

I have heard that Random Forest and other tree based machines apply some kind of implicit feature selection. My Question is: Does this also apply for machines like the SVM? As far as I understand is ...
0
votes
0answers
14 views

How to get subsample indices from sklearn.BaggingRegressor

I'm trying to define the number of the repeated samples in sklearn random forest subsamples: Here is my code: ...
1
vote
1answer
22 views

How to train a Machine Learning model for blocked data

I'm concerned with a supervised classification problem for the following type of data. The data consists of $N$ rows (where $N$ is not very large - this is not a big-data problem) and $M$ columns (...
0
votes
0answers
14 views

Using selected variables after dimensionality reduction throws a value error

I am working on a regression problem, namely the Boston House prediction problem hosted on Kaggle. I am currently using Random Forest Classifier to reduce the dimensions of my dataset. But right now, ...
0
votes
1answer
19 views

Adding time as a feature with xgboost/random forests

I am trying to use xgboost for performing some regression and the features I have are rather simple and limited. I have the time stamp associated with some ...
2
votes
1answer
93 views

Feature Importance

I have a dataset with 10 features. I've computed the feature importance using permutation importance with cross-validation from eli5, after fitting an extremely randomized trees (ET) classifier form ...
0
votes
2answers
41 views

Model should predict the same value every time for the same input

I have used a random forest model for prediction of prices. Should the model be predictable in its behavior? By this, I mean that I'm not changing the model and the input, Will the predicted value ...
1
vote
1answer
63 views

Can I arbitrarily eliminate 20% of my training data if doing so significantly improves model accuracy?

My dataset contains 2000 records with 125 meaningful fields 5 of which are distributed along highly skewed lognormal behavior. I've found that if I eliminate all records below some threshold of this ...
3
votes
2answers
55 views

Which classifier performs better when using 'class_weight'?

I have used the 'class_weight' method to balance my multi-class classification problem, using Logistic Regression, Random Forest, and XGBoost classifiers. Among these three methods, logistic ...
1
vote
0answers
88 views

Evaluating Random Forest regression model that predicts low values for skewed dependent variable

Background I'm trying to predict the value of website visitors. Only a small fraction of the visitors actually make a purchase, so ~97% of the visits has the value of 0, while about 2-3% has values ...
0
votes
1answer
67 views

Using a LinearSVC() for multilabel classification with MultiOutputClassifier() in a pipeline in sci-kit learn

My input data is a (23948,) pandas.Series of strings containing newspaper headlines. My target are 20 labels of the headline (e.g. 'crime', 'politics') each binarily encoded with [0, 1]. The labels ...
1
vote
1answer
25 views

How do we decide on the classification algorithm to use with huge training size?

I am solving a questions binary classification problem and the training size for this is huge(291 billion). The data has bloated because of using tfidfvectorizerfor ...
2
votes
2answers
72 views

Calculate future GDP % using machine learning

I need to estimate the GDP % of a country three years into the future, based on historic data. I have 30+ years of the following monthly data that includes features such as inflation and unemployment ...
2
votes
1answer
30 views

Force selecting samples in majority class with random forest

Context: I have some data to fit a random forest classifier (binary output) with 1 being a very rare event. In particular, in my training set, there are only 614 1's out of 29400 points. I am using ...
1
vote
1answer
84 views

Isolation forest - grouped by

I'm trying to use isolation forest algorithm for outliers detection. Data has 2 columns: id and REV. Below code gives me ...
0
votes
1answer
22 views

Is there a case that random forest data (a bunch of trees) consumes MB or GB of memory?

Is there any case that random forest (a bunch of trees) consumes too much memory in practice? I'm wondering why my scikit-learn consumes large amount of memory.
0
votes
0answers
12 views

What is the default method for Orange to deal with missing values in random forest classification?

Good day, I have built a random forest classification model in Orange, but some of my input data (all continuous) are missing. Up to 30% of the data for some variables are missing. I understand the ...
2
votes
1answer
35 views

Significant drop from validation accuracy to test accuracy

I am more familiar with classification tasks, though I have been working on a regression problem. I was given a large training dataset (>70k samples) and an independently collected test set (~2k). I ...
1
vote
2answers
42 views

ScikitLearn - RandomForestRegressor score different in and out of grid search

I am using RandomForestRegressor (scikit-learn python package). I am looking for the best values for hyperparameters ...
2
votes
2answers
94 views

Using random forest for selecting variables returns the entire dataframe

I am in the process of dimensionality reduction. I am using Random Forest to find the columns with the highest level of correlation with the target SalePrice column. The problem is that the output ...
2
votes
2answers
67 views

Decision Trees - how does split for categorical features happen?

A decision tree, while performing recursive binary splitting, selects an independent variable (say $X_j$) and a threshold (say $t$) such that the predictor space is split into regions {$X|X_j < t$} ...
1
vote
1answer
13 views

clarification on splitting individual trees in extra trees?

So I am a beginner in machine learning and just started learning about random trees in this article here. When it talks about tuning the hyperparameter K, I'm a bit confused as to how it works. It ...
1
vote
1answer
94 views

Choosing weights on random forest for imbalanced data with the aim to minimize false positives

I am currently dealing with a binary classification task on imbalanced data with the following distribution: ...
0
votes
1answer
23 views

How do I interpret the given classification report?

The given classification report was obtained from running a Random Forest binary classifier on the test data. There is huge class imbalance in the training data. How do I interpret the given ...