# Questions tagged [random-forest]

Random forest is a machine learning ensemble method based on choosing random subsets of observations and variables for each of many decision trees.

213 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6 votes
1 answer
4k views

### How does XGBoost compute the probabilities in predict_proba()?

I'm using the sklearn wrapper for XGBoost. I didn't manage to find a clear explanation for the way the probabilities given as output by predict_proba() are computed....
• 61
5 votes
1 answer
503 views

### Illustrating the dimensionality reduction done by a classification or regression model

Tl;DR: You can predict something, but how do you explain the prediction? EDIT: I have built a website that tries to answer this question with means of embedding / visually clustering data according ...
• 151
4 votes
1 answer
247 views

### How important is lookahead search in decision trees?

I am using random forests, and, in my data, I have a lot of situations where $X_1$ is a bad predictor, $X_2$ is a bad predictor, but the joint distribution would make a good predictor. Say that $X1$, ...
• 3,420
3 votes
0 answers
171 views

### Non-greedy decision tree / random forest implementation(s) in Python

The standard random forest is trained using a greedy approach for computational feasibility. However, there are a number of alternative methods such as "lookahead" or using bilevel ...
• 7,456
3 votes
1 answer
133 views

### ML model to forecast time series data

This question has three sub-parts, answering each of which probably doesn't require huge text. I hope that is okay. I'm trying to understand time series prediction using ML. I have the target variable ...
• 167
3 votes
0 answers
274 views

• 226
2 votes
0 answers
1k views

### What is the difference between PySpark's featuresCol, labelCol, predictionCol, and probabilityCol?

I am attempting to train a random forest classifier (pyspark.ml.classification.RandomForestClassifier) on a large dataset (~70gb). However, I am not sure what to ...
• 322
2 votes
0 answers
323 views

### Wrong train/test split strategy

The question is about a wrongly chosen strategy for train/test splitting in a RandomForest model. I know choosing the test set this way gives the wrong output but I would like to understand why. (The ...
• 221
2 votes
0 answers
2k views

### Code for Multivariate Random Forest in Python/R?

I am trying to code multivariate (or Multi output dx input features and dy outputs) Random Forest Regressor algorithm for a project, i.e. the algorithm can be used to predict more than one dependent ...
• 21
2 votes
0 answers
511 views

### randomForest::varImp VS conditional variable importance

Data: My training set consists of ~450k obs and 26 variables, out of which 1 is an ordinal factor (order_month, 12 levels) and the rest is numerical. Moreover, some of my predictors are highly ...
• 193
2 votes
0 answers
96 views

### what predictive analysis will work with this data set?

I have a dataset that includes all of the building permits that were issued for homes within a city. I have the data in a "snapshot" from a year ago (and potentially a few others, older snapshots) and ...
• 121
2 votes
1 answer
96 views

### Accept any suggestion to create training data from correlation matrix to find odd one out to identify difference in variation

I have N time varying feature vectors obtained by recording different parameters over time.This results in N*N similarity matrix which contains one to one correlations value for each feature. We need ...
2 votes
0 answers
127 views

### Random Forest Class Weighting for Logistic Probabilities

I have a model at work that I am building and am running into some odd outputs from the random forest as it pertains to the probability of response. In my case, the class distributions are very ...
2 votes
0 answers
4k views

### Error::Type of predictors in new data do not match that of the training data

I am building a classification model using randomForest. When trying to predict I get the below error Type of predictors in new data do not match that of the ...
• 717
1 vote
0 answers
37 views

### How do I know the appropriate number of iterations when using Miceforest for imputation?

I want to know how to avoid overfitting without having to increase the number of iterations excessively in Python with the Miceforest library. I know you can make a correlation map of data sets but I ...
1 vote
0 answers
90 views

### Feature importance using random forest vs. SHAP

I recently came across SHAP while looking for feature-importance methods. To use SHAP, first a model needs to be created, and then based on the predictions made by the model, SHAP values are ...
• 183
1 vote
0 answers
29 views

### How to know which rules were applied to predict one sample in trained decision tree model?

I have trained Random Forest Regressor from sklearn. I am able to return text representation from each Decision Tree rule using tree.export_text (sklearn documentation here). But it shows rules for ...
• 15
1 vote
0 answers
7 views

### Huge variance for RandomForestRegressor models

The experiment is the following: train a RFR with a 15k train rows get predictions on 8k test rows, save predictions as y_hat0 remove 1 random row from the training set and retrain the RFR save ...
• 11
1 vote
0 answers
42 views

### Can you do a power analysis to determine the sample size for a virtual species simulation which is modeled using random forests?

I am simulating virtual species on a 100x100 grid (the size for now). Each grid layer represents one environmental variable. The "suitability function" defines the probability of a presence ...
• 111
1 vote
1 answer
31 views

### Calculating the importance metric in random forest: Why don't we remove the variable instead of permutating its values?

The importance metric in random forests is a way to determine the significance of a predictor variable in a model. It does this by randomly permutating the values of one predictor variable at a time ...
• 121
1 vote
0 answers
223 views

### Sklearn predicts different results depending on the input length

Here is the problem: I fitted a Random Forest Classifier and saved it to a pickle file. However, when I predict with the entire dataset I get one result, and when run predict line by line (loop) I get ...
• 11
1 vote
1 answer
28 views

### Random forest - estimate range instead of exact value

I was wondering whether one could adjust a random forest to estimate a range of values instead of receiving one exact estimate. What I mean by that: my current rf predicts a value of e.g. 5 based on ...
• 11
1 vote
2 answers
21 views

### Random Forest with less samples & variation in test_scores

I'm building a RandomForestRegressor with 75 samples. The distribution of y (After train_test_split) is as below. (Blue-Train and Red-Test) Keeping test_size=0.3 (...
• 143
1 vote
2 answers
240 views

### Is there a point in hyperparameter tuning for Random Forests?

I have a binary classification task with substantial class imbalance (99% negative - 1% positive). I want to developed a Random Forest model to make prediction, and after establishing a baseline (with ...
• 11
1 vote
1 answer
602 views

### How to predict on data that is label encoded as end user will input a categorical data?

My dataset contains about 29 features with 3 class labels as result. Among these 29 features around 24 features are categorical i cannot transform each category into numbers as there are many more ...
1 vote
0 answers
108 views

### Why is GPU accelerated node much slower than CPU node for training a random forest model on databricks?

I have a dataset about 5 million rows with 14 features and a binary target. I decided to train a pyspark random forest classifier on Databricks. The CPU cluster I created contains 2 c4.8xlarge workers ...
1 vote
0 answers
26 views

### Consequence of having different factor levels in production after training random forest model in R (wrong variable language encoding before)

I just realised that a model that is going in production had an issue with encoding for a specific variable. How does a random forest deal with the fact that: The dataset used for training, validation ...
• 11
1 vote
0 answers
8 views

### Precision and AUROC for which class values

I am a newbie in reading research paper and implementing it by myself. I went through the paper Breast Cancer Survival Prediction from Imbalanced Dataset with Machine Learning Algorithms. Can anyone ...
• 359
1 vote
1 answer
29 views

### How to find a measurable indicator of a condition using random forest

I have data for 50 patients. This data comprises a different set of variables, of which one of them is a binary variable. For example, the presence of a given symptom (1= have the symptom, 0 = doesn't ...
• 11
1 vote
0 answers
34 views

### Random Forest Generating Bad Predictions: What might the issue be?

I'm using sklearn's RandomForestRegressor to try and model a relationship that involves three Feature variables (x1,x2,x3) and ...