Questions tagged [random-forest]
Random forest is a machine learning ensemble method based on choosing random subsets of observations and variables for each of many decision trees.
222
questions with no upvoted or accepted answers
6
votes
1
answer
2k
views
How does XGBoost compute the probabilities in predict_proba()?
I'm using the sklearn wrapper for XGBoost.
I didn't manage to find a clear explanation for the way the probabilities given as output by predict_proba() are computed....
- 61
5
votes
1
answer
486
views
Illustrating the dimensionality reduction done by a classification or regression model
Tl;DR: You can predict something, but how do you explain the prediction?
EDIT: I have built a website that tries to answer this question with means of embedding / visually clustering data according ...
- 151
4
votes
2
answers
83
views
Preventing fitting Regression CNN to the mean when dataset has only few outliers
I am trying to train a CNN for regression on a dataset where most of the points lie around a similar output value. There are however a few outliers that are very important but they are less ...
- 141
4
votes
1
answer
217
views
How important is lookahead search in decision trees?
I am using random forests, and, in my data, I have a lot of situations where $X_1$ is a bad predictor, $X_2$ is a bad predictor, but the joint distribution would make a good predictor.
Say that $X1$, ...
- 3,360
3
votes
0
answers
123
views
Non-greedy decision tree / random forest implementation(s) in Python
The standard random forest is trained using a greedy approach for computational feasibility. However, there are a number of alternative methods such as "lookahead" or using bilevel ...
- 7,247
3
votes
1
answer
95
views
ML model to forecast time series data
This question has three sub-parts, answering each of which probably doesn't require huge text. I hope that is okay.
I'm trying to understand time series prediction using ML. I have the target variable ...
- 157
3
votes
0
answers
209
views
Unbiasedness of random forests
Suppose that I am trying to build a random forest by subsampling the data and choosing a single feature per tree randomly. For example, suppose there is some dataset,
$D = \{(x_{1},y_{1}), ......(x_{N}...
- 131
3
votes
2
answers
654
views
Why does my random forest classifier predicts one class more often?
I have a random forest classifier that predicts 0 class about twice as often as class 1. It also predicts class 0 with higher probabilities than class 1.
It is not a imbalanced dataset.
I tried ...
- 31
3
votes
0
answers
122
views
Initial value space for Random Forest hyperparameter tuning
I'm building a Random Forest Classifier using Scikit Learn.
My problem consists in a 4 class classification task, the values are distributed as follows (after splitting my data in training set and ...
3
votes
0
answers
34
views
Serializing a trained classification model into a set of actionable insights
I'm looking for ways to convert a trained classification model into a list of insights based on the resulting parameters of the model.
To make an example, let's assume we trained a decision tree to ...
- 63
3
votes
0
answers
749
views
Target transformation for tree models
Can anybody explain why/if target variable transformations could help when dealing with tree based models?
I've seen this excellent reply which explains quite well why it shouldn't affect if ...
- 241
3
votes
3
answers
163
views
Overfitted model produces similar AUC on test set, so which model do I go with?
I was trying to compare the effect of running GridSearchCV on a dataset which was oversampled prior and oversampled after the training folds are selected. The oversampling approach I used was random ...
- 351
3
votes
1
answer
2k
views
IsolationForest Decision Function vs. Anomaly Prediction Question
I'm currently working on an unsupervised anomaly detection project, and for it I'm using IsolationForest through scikit-learn. My question is, why/how is it possible for the model to predict something ...
- 41
3
votes
1
answer
256
views
Does CART algorithm takes into account in the order of the set of attributes?
when using matlab command 'fitctree' for classification purpose, and I change the order of the attributes I do not find the same Tree and thus the same classificaiton error? why? CART algorithm does ...
- 43
3
votes
0
answers
76
views
Various algorithms performance in a problem and what can be deduced about data and problem?
HI I am currently trying to apply various algorithms to a classification problem to assess which could be better and then try to fine tune the bests of the first approach. I am a beginner so I use ...
- 131
2
votes
2
answers
116
views
How to interpret importance of random forest model, Mean Decrease Accuracy and Mean Decrease Gini?
A random forest model outputs the following importance values. How do I interpert them for feature selection? If it's the mean decreased accuracy does that mean that by removing them from the model ...
- 21
2
votes
0
answers
565
views
Sample size for SHAP explainer and range of a SHAP value
I am working on a binary classification with 977 records with 77:23 class proportion. I used random forest model.
Based on my attempt to run SHAP package, I got the below plots
And I also see that ...
- 2,507
2
votes
1
answer
1k
views
What is the difference between RandomForestClassifier and XGBRFClassifier?
What is the difference between RandomForestClassifier and XGBRFClassifier?
There is no detailed explanation about what XGBRFClassifier exactly is so I was wondering.
- 1,027
2
votes
0
answers
26
views
Shifting The Result of Regression Model By N-Value
I am doing a multivariable regression problem, which predicts the frequency of a failure mode in a production system. In this problem, I used XGBRF for Regression as my ML model.
These are the results ...
- 21
2
votes
1
answer
28
views
Random Forest Regression Analysis - Comprehension problem
I have an deeply understanding-problem with Random Forest Regression. Target is a university project: We have to do a random forest regression analytics with financial data in R.
I already read many ...
- 21
2
votes
0
answers
53
views
Shuffling data yields significantly worse performance
Edit: I've experimented a few times, shuffling the data at various steps. It seems that as long as I restart the python kernel and reset the dataframe indices, the performance is good. I'm still not ...
- 63
2
votes
1
answer
53
views
Out-of-bag error in Orange
There is some way to see the out-of-bag error in Orange? The widget Test and Score only reports mean squared error, root mean square, mean absolute error, coefficient of determination and root-mean-...
- 21
2
votes
1
answer
131
views
How do i create a classifier on sensor data?
I am working on a indoor localization based on magnetometer.
I have 9 separate time-series datasets of sensor readings taken from coordinates 00, 01, 02, 10, 11, and so on until 22. Basically I am ...
- 143
2
votes
0
answers
22
views
Negative correlation between OOB statistics and test set statistics during tuning of a RandomForest
I am tuning the parameters of a binary random forest classifier using a random search with a priority queue for training. After training with a fixed number of estimators (3000), the strategy is to ...
- 69
2
votes
1
answer
145
views
How to apply model to training data to identify mislabeled observations?
I have a list of people, attributes about those people (height, weight, blood pressure, etc.), and a binary target variable called has_heart_issues. This data ...
2
votes
0
answers
129
views
When does random forest feature importance fail?
I'm curious about the assumptions of random forest feature importance.
In this paper, the author says that
"We show that random forest variable importance measures are a sensible means for ...
- 139
2
votes
0
answers
39
views
Rescale prediction to correct dispersion due to correlation between response and residuals in Random Forrest Regression
I am using Random Forest Regression and I observe a strong positive correlation between the residuals ($\hat{u}$) and the response variable ($y$) which lead to a dispersion : predictions are ...
- 226
2
votes
1
answer
200
views
Dataset where svm performance is significantly different from random forest
Is there a specific dataset where svm performs significantly better or worse than random forest?
I know that the performance could depend on the dataset but is there a specific dataset?
2
votes
0
answers
1k
views
What is the difference between PySpark's featuresCol, labelCol, predictionCol, and probabilityCol?
I am attempting to train a random forest classifier (pyspark.ml.classification.RandomForestClassifier) on a large dataset (~70gb). However, I am not sure what to ...
- 312
2
votes
1
answer
120
views
How we can have RF-QLearning or SVR-QLearning (Combine these algorithm with a Q-Learning )
How we can have RF-QLearning or SVR-QLearning (Combine these algorithm with a Q-Learning )?
I want to replace the DNN section of Qlearning with a RF or SVR but the problem is that there is no clear ...
- 1,754
2
votes
0
answers
319
views
Wrong train/test split strategy
The question is about a wrongly chosen strategy for train/test splitting in a RandomForest model. I know choosing the test set this way gives the wrong output but I would like to understand why.
(The ...
- 221
2
votes
0
answers
2k
views
Code for Multivariate Random Forest in Python/R?
I am trying to code multivariate (or Multi output dx input features and dy outputs) Random Forest Regressor algorithm for a project, i.e. the algorithm can be used to predict more than one dependent ...
- 21
2
votes
0
answers
505
views
randomForest::varImp VS conditional variable importance
Data:
My training set consists of ~450k obs and 26 variables, out of which 1 is an ordinal factor (order_month, 12 levels) and the rest is numerical. Moreover, some of my predictors are highly ...
- 193
2
votes
0
answers
95
views
what predictive analysis will work with this data set?
I have a dataset that includes all of the building permits that were issued for homes within a city. I have the data in a "snapshot" from a year ago (and potentially a few others, older snapshots) and ...
- 121
2
votes
1
answer
90
views
Accept any suggestion to create training data from correlation matrix to find odd one out to identify difference in variation
I have N time varying feature vectors obtained by recording different parameters over time.This results in N*N similarity matrix which contains one to one correlations value for each feature. We need ...
- 121
2
votes
0
answers
126
views
Random Forest Class Weighting for Logistic Probabilities
I have a model at work that I am building and am running into some odd outputs from the random forest as it pertains to the probability of response. In my case, the class distributions are very ...
- 121
2
votes
0
answers
4k
views
Error::Type of predictors in new data do not match that of the training data
I am building a classification model using randomForest. When trying to predict I get the below error
Type of predictors in new data do not match that of the ...
- 717
2
votes
1
answer
360
views
How do I deal with non-IID data in gradient boosted random forest (for stock market)?
I am working on a stock market decision system. I have currently centered on gradient boosting as the likely best machine learning solution for the problem.
However, I have 2 fundamental issues with ...
1
vote
0
answers
31
views
Can you do a power analysis to determine the sample size for a virtual species simulation which is modeled using random forests?
I am simulating virtual species on a 100x100 grid (the size for now). Each grid layer represents one environmental variable. The "suitability function" defines the probability of a presence ...
- 111
1
vote
1
answer
25
views
Random forest - estimate range instead of exact value
I was wondering whether one could adjust a random forest to estimate a range of values instead of receiving one exact estimate. What I mean by that: my current rf predicts a value of e.g. 5 based on ...
- 11
1
vote
2
answers
19
views
Random Forest with less samples & variation in test_scores
I'm building a RandomForestRegressor with 75 samples. The distribution of y (After train_test_split) is as below. (Blue-Train and Red-Test)
Keeping test_size=0.3 (...
- 133
1
vote
2
answers
93
views
Is there a point in hyperparameter tuning for Random Forests?
I have a binary classification task with substantial class imbalance (99% negative - 1% positive). I want to developed a Random Forest model to make prediction, and after establishing a baseline (with ...
- 11
1
vote
1
answer
249
views
How to predict on data that is label encoded as end user will input a categorical data?
My dataset contains about 29 features with 3 class labels as result. Among these 29 features around 24 features are categorical i cannot transform each category into numbers as there are many more ...
1
vote
0
answers
47
views
Why is GPU accelerated node much slower than CPU node for training a random forest model on databricks?
I have a dataset about 5 million rows with 14 features and a binary target. I decided to train a pyspark random forest classifier on Databricks. The CPU cluster I created contains 2 c4.8xlarge workers ...
- 13
1
vote
0
answers
18
views
Consequence of having different factor levels in production after training random forest model in R (wrong variable language encoding before)
I just realised that a model that is going in production had an issue with encoding for a specific variable. How does a random forest deal with the fact that:
The dataset used for training, validation ...
- 11
1
vote
0
answers
8
views
Precision and AUROC for which class values
I am a newbie in reading research paper and implementing it by myself. I went through the paper Breast Cancer Survival Prediction from Imbalanced Dataset with Machine Learning Algorithms.
Can anyone ...
- 359
1
vote
1
answer
25
views
How to find a measurable indicator of a condition using random forest
I have data for 50 patients.
This data comprises a different set of variables, of which one of them is a binary variable. For example, the presence of a given symptom (1= have the symptom, 0 = doesn't ...
- 11
1
vote
0
answers
27
views
Random Forest Generating Bad Predictions: What might the issue be?
I'm using sklearn's RandomForestRegressor to try and model a relationship that involves three Feature variables (x1,x2,x3) and ...
1
vote
1
answer
203
views
Automated feature selection - Best practice to avoid data leakage?
This question relates generally to all automated feature selection approaches. In my particular scenario, we have a python package called tsfresh and multiclass classification.
What has been done so ...
- 39
1
vote
0
answers
20
views
Any way to represent a random forest regressor model?
currently doing some EDA into a random forest regressor that was built; there seems to be observations where the model prediction is off. what library can i use to visualise the representation of the ...
- 11