Questions tagged [random-forest]

Random forest is a machine learning ensemble method based on choosing random subsets of observations and variables for each of many decision trees.

213 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6 votes
1 answer
4k views

How does XGBoost compute the probabilities in predict_proba()?

I'm using the sklearn wrapper for XGBoost. I didn't manage to find a clear explanation for the way the probabilities given as output by predict_proba() are computed....
keren42's user avatar
  • 61
5 votes
1 answer
503 views

Illustrating the dimensionality reduction done by a classification or regression model

Tl;DR: You can predict something, but how do you explain the prediction? EDIT: I have built a website that tries to answer this question with means of embedding / visually clustering data according ...
BenoitParis's user avatar
4 votes
1 answer
247 views

How important is lookahead search in decision trees?

I am using random forests, and, in my data, I have a lot of situations where $X_1$ is a bad predictor, $X_2$ is a bad predictor, but the joint distribution would make a good predictor. Say that $X1$, ...
Ricardo Cruz's user avatar
  • 3,420
3 votes
0 answers
171 views

Non-greedy decision tree / random forest implementation(s) in Python

The standard random forest is trained using a greedy approach for computational feasibility. However, there are a number of alternative methods such as "lookahead" or using bilevel ...
Peter's user avatar
  • 7,456
3 votes
1 answer
133 views

ML model to forecast time series data

This question has three sub-parts, answering each of which probably doesn't require huge text. I hope that is okay. I'm trying to understand time series prediction using ML. I have the target variable ...
user9343456's user avatar
3 votes
0 answers
274 views

Unbiasedness of random forests

Suppose that I am trying to build a random forest by subsampling the data and choosing a single feature per tree randomly. For example, suppose there is some dataset, $D = \{(x_{1},y_{1}), ......(x_{N}...
user1234's user avatar
  • 131
3 votes
2 answers
991 views

Why does my random forest classifier predicts one class more often?

I have a random forest classifier that predicts 0 class about twice as often as class 1. It also predicts class 0 with higher probabilities than class 1. It is not a imbalanced dataset. I tried ...
Ondřej Vitík's user avatar
3 votes
0 answers
144 views

Initial value space for Random Forest hyperparameter tuning

I'm building a Random Forest Classifier using Scikit Learn. My problem consists in a 4 class classification task, the values are distributed as follows (after splitting my data in training set and ...
Mattia Surricchio's user avatar
3 votes
0 answers
40 views

Serializing a trained classification model into a set of actionable insights

I'm looking for ways to convert a trained classification model into a list of insights based on the resulting parameters of the model. To make an example, let's assume we trained a decision tree to ...
ozz1k's user avatar
  • 83
3 votes
0 answers
862 views

Target transformation for tree models

Can anybody explain why/if target variable transformations could help when dealing with tree based models? I've seen this excellent reply which explains quite well why it shouldn't affect if ...
Ludecan's user avatar
  • 261
3 votes
3 answers
181 views

Overfitted model produces similar AUC on test set, so which model do I go with?

I was trying to compare the effect of running GridSearchCV on a dataset which was oversampled prior and oversampled after the training folds are selected. The oversampling approach I used was random ...
rayven1lk's user avatar
  • 371
3 votes
1 answer
273 views

Does CART algorithm takes into account in the order of the set of attributes?

when using matlab command 'fitctree' for classification purpose, and I change the order of the attributes I do not find the same Tree and thus the same classificaiton error? why? CART algorithm does ...
LSola's user avatar
  • 43
3 votes
0 answers
77 views

Various algorithms performance in a problem and what can be deduced about data and problem?

HI I am currently trying to apply various algorithms to a classification problem to assess which could be better and then try to fine tune the bests of the first approach. I am a beginner so I use ...
Ando Jurai's user avatar
2 votes
0 answers
249 views

Does it make sense to use target encoding together with tree-based models?

I'm working on a regression problem with a few high-cardinality categorical features (Forecasting different items with a single model). Someone suggested to use target-encoding (mean/median of the ...
KJA's user avatar
  • 21
2 votes
2 answers
366 views

How to interpret importance of random forest model, Mean Decrease Accuracy and Mean Decrease Gini?

A random forest model outputs the following importance values. How do I interpert them for feature selection? If it's the mean decreased accuracy does that mean that by removing them from the model ...
FOH's user avatar
  • 21
2 votes
0 answers
899 views

Sample size for SHAP explainer and range of a SHAP value

I am working on a binary classification with 977 records with 77:23 class proportion. I used random forest model. Based on my attempt to run SHAP package, I got the below plots And I also see that ...
The Great's user avatar
  • 2,565
2 votes
0 answers
55 views

Shifting The Result of Regression Model By N-Value

I am doing a multivariable regression problem, which predicts the frequency of a failure mode in a production system. In this problem, I used XGBRF for Regression as my ML model. These are the results ...
Lacrin's user avatar
  • 21
2 votes
1 answer
33 views

Random Forest Regression Analysis - Comprehension problem

I have an deeply understanding-problem with Random Forest Regression. Target is a university project: We have to do a random forest regression analytics with financial data in R. I already read many ...
Hopper's user avatar
  • 21
2 votes
0 answers
60 views

Shuffling data yields significantly worse performance

Edit: I've experimented a few times, shuffling the data at various steps. It seems that as long as I restart the python kernel and reset the dataframe indices, the performance is good. I'm still not ...
tensormoby's user avatar
2 votes
1 answer
65 views

Out-of-bag error in Orange

There is some way to see the out-of-bag error in Orange? The widget Test and Score only reports mean squared error, root mean square, mean absolute error, coefficient of determination and root-mean-...
kvratto's user avatar
  • 21
2 votes
1 answer
178 views

How do i create a classifier on sensor data?

I am working on a indoor localization based on magnetometer. I have 9 separate time-series datasets of sensor readings taken from coordinates 00, 01, 02, 10, 11, and so on until 22. Basically I am ...
harry r's user avatar
  • 153
2 votes
0 answers
24 views

Negative correlation between OOB statistics and test set statistics during tuning of a RandomForest

I am tuning the parameters of a binary random forest classifier using a random search with a priority queue for training. After training with a fixed number of estimators (3000), the strategy is to ...
Net_Raider's user avatar
2 votes
0 answers
61 views

Understanding the output of the Random Forest method for classification

I'm using a Random Forest method to predict the behavior of failures at Period_12. My dataset has information about the eleven periods before, considering 112 subperiods (rows). Each one of these ...
Fernanda's user avatar
2 votes
1 answer
183 views

How to apply model to training data to identify mislabeled observations?

I have a list of people, attributes about those people (height, weight, blood pressure, etc.), and a binary target variable called has_heart_issues. This data ...
OverflowingTheGlass's user avatar
2 votes
0 answers
135 views

When does random forest feature importance fail?

I'm curious about the assumptions of random forest feature importance. In this paper, the author says that "We show that random forest variable importance measures are a sensible means for ...
Eric Kim's user avatar
  • 149
2 votes
0 answers
45 views

Rescale prediction to correct dispersion due to correlation between response and residuals in Random Forrest Regression

I am using Random Forest Regression and I observe a strong positive correlation between the residuals ($\hat{u}$) and the response variable ($y$) which lead to a dispersion : predictions are ...
SoufianeK's user avatar
  • 226
2 votes
0 answers
1k views

What is the difference between PySpark's featuresCol, labelCol, predictionCol, and probabilityCol?

I am attempting to train a random forest classifier (pyspark.ml.classification.RandomForestClassifier) on a large dataset (~70gb). However, I am not sure what to ...
Joe B's user avatar
  • 322
2 votes
0 answers
323 views

Wrong train/test split strategy

The question is about a wrongly chosen strategy for train/test splitting in a RandomForest model. I know choosing the test set this way gives the wrong output but I would like to understand why. (The ...
DBSE's user avatar
  • 221
2 votes
0 answers
2k views

Code for Multivariate Random Forest in Python/R?

I am trying to code multivariate (or Multi output dx input features and dy outputs) Random Forest Regressor algorithm for a project, i.e. the algorithm can be used to predict more than one dependent ...
Garvit's user avatar
  • 21
2 votes
0 answers
511 views

randomForest::varImp VS conditional variable importance

Data: My training set consists of ~450k obs and 26 variables, out of which 1 is an ordinal factor (order_month, 12 levels) and the rest is numerical. Moreover, some of my predictors are highly ...
Kasia Kulma's user avatar
2 votes
0 answers
96 views

what predictive analysis will work with this data set?

I have a dataset that includes all of the building permits that were issued for homes within a city. I have the data in a "snapshot" from a year ago (and potentially a few others, older snapshots) and ...
jonmrich's user avatar
  • 121
2 votes
1 answer
96 views

Accept any suggestion to create training data from correlation matrix to find odd one out to identify difference in variation

I have N time varying feature vectors obtained by recording different parameters over time.This results in N*N similarity matrix which contains one to one correlations value for each feature. We need ...
Nikita Chopra's user avatar
2 votes
0 answers
127 views

Random Forest Class Weighting for Logistic Probabilities

I have a model at work that I am building and am running into some odd outputs from the random forest as it pertains to the probability of response. In my case, the class distributions are very ...
Nate Thompson's user avatar
2 votes
0 answers
4k views

Error::Type of predictors in new data do not match that of the training data

I am building a classification model using randomForest. When trying to predict I get the below error Type of predictors in new data do not match that of the ...
Arun's user avatar
  • 717
1 vote
0 answers
37 views

How do I know the appropriate number of iterations when using Miceforest for imputation?

I want to know how to avoid overfitting without having to increase the number of iterations excessively in Python with the Miceforest library. I know you can make a correlation map of data sets but I ...
Eduardo Dimas's user avatar
1 vote
0 answers
90 views

Feature importance using random forest vs. SHAP

I recently came across SHAP while looking for feature-importance methods. To use SHAP, first a model needs to be created, and then based on the predictions made by the model, SHAP values are ...
lsr729's user avatar
  • 183
1 vote
0 answers
29 views

How to know which rules were applied to predict one sample in trained decision tree model?

I have trained Random Forest Regressor from sklearn. I am able to return text representation from each Decision Tree rule using tree.export_text (sklearn documentation here). But it shows rules for ...
Paulina's user avatar
  • 15
1 vote
0 answers
7 views

Huge variance for RandomForestRegressor models

The experiment is the following: train a RFR with a 15k train rows get predictions on 8k test rows, save predictions as y_hat0 remove 1 random row from the training set and retrain the RFR save ...
Oleg's user avatar
  • 11
1 vote
0 answers
42 views

Can you do a power analysis to determine the sample size for a virtual species simulation which is modeled using random forests?

I am simulating virtual species on a 100x100 grid (the size for now). Each grid layer represents one environmental variable. The "suitability function" defines the probability of a presence ...
Kyle's user avatar
  • 111
1 vote
1 answer
31 views

Calculating the importance metric in random forest: Why don't we remove the variable instead of permutating its values?

The importance metric in random forests is a way to determine the significance of a predictor variable in a model. It does this by randomly permutating the values of one predictor variable at a time ...
Asterion's user avatar
  • 121
1 vote
0 answers
223 views

Sklearn predicts different results depending on the input length

Here is the problem: I fitted a Random Forest Classifier and saved it to a pickle file. However, when I predict with the entire dataset I get one result, and when run predict line by line (loop) I get ...
Juarez's user avatar
  • 11
1 vote
1 answer
28 views

Random forest - estimate range instead of exact value

I was wondering whether one could adjust a random forest to estimate a range of values instead of receiving one exact estimate. What I mean by that: my current rf predicts a value of e.g. 5 based on ...
noels's user avatar
  • 11
1 vote
2 answers
21 views

Random Forest with less samples & variation in test_scores

I'm building a RandomForestRegressor with 75 samples. The distribution of y (After train_test_split) is as below. (Blue-Train and Red-Test) Keeping test_size=0.3 (...
Salih's user avatar
  • 143
1 vote
2 answers
240 views

Is there a point in hyperparameter tuning for Random Forests?

I have a binary classification task with substantial class imbalance (99% negative - 1% positive). I want to developed a Random Forest model to make prediction, and after establishing a baseline (with ...
BoS_88's user avatar
  • 11
1 vote
1 answer
602 views

How to predict on data that is label encoded as end user will input a categorical data?

My dataset contains about 29 features with 3 class labels as result. Among these 29 features around 24 features are categorical i cannot transform each category into numbers as there are many more ...
Muhammad Minhas's user avatar
1 vote
0 answers
108 views

Why is GPU accelerated node much slower than CPU node for training a random forest model on databricks?

I have a dataset about 5 million rows with 14 features and a binary target. I decided to train a pyspark random forest classifier on Databricks. The CPU cluster I created contains 2 c4.8xlarge workers ...
Zhenyu Zhang's user avatar
1 vote
0 answers
26 views

Consequence of having different factor levels in production after training random forest model in R (wrong variable language encoding before)

I just realised that a model that is going in production had an issue with encoding for a specific variable. How does a random forest deal with the fact that: The dataset used for training, validation ...
Andy's user avatar
  • 11
1 vote
0 answers
8 views

Precision and AUROC for which class values

I am a newbie in reading research paper and implementing it by myself. I went through the paper Breast Cancer Survival Prediction from Imbalanced Dataset with Machine Learning Algorithms. Can anyone ...
Encipher's user avatar
  • 359
1 vote
1 answer
29 views

How to find a measurable indicator of a condition using random forest

I have data for 50 patients. This data comprises a different set of variables, of which one of them is a binary variable. For example, the presence of a given symptom (1= have the symptom, 0 = doesn't ...
roybatty's user avatar
1 vote
0 answers
34 views

Random Forest Generating Bad Predictions: What might the issue be?

I'm using sklearn's RandomForestRegressor to try and model a relationship that involves three Feature variables (x1,x2,x3) and ...
Austin Prater's user avatar

1
2 3 4 5