Questions tagged [random-forest]

Random forest is a machine-learning classifier based on choosing random subsets of variables for each tree and using the most frequent tree output as the overall classification.

Filter by
Sorted by
Tagged with
0
votes
0answers
1 view

Implicit feature selection

I have heard that Random Forest and other tree based machines apply some kind of implicit feature selection. My Question is: Does this also apply for machines like the SVM? As far as I understand is ...
0
votes
0answers
9 views

How to get subsample indices from sklearn.BaggingRegressor

I'm trying to define the number of the repeated samples in sklearn random forest subsamples: Here is my code: ...
1
vote
1answer
16 views

How to train a Machine Learning model for blocked data

I'm concerned with a supervised classification problem for the following type of data. The data consists of $N$ rows (where $N$ is not very large - this is not a big-data problem) and $M$ columns (...
0
votes
0answers
14 views

Using selected variables after dimensionality reduction throws a value error

I am working on a regression problem, namely the Boston House prediction problem hosted on Kaggle. I am currently using Random Forest Classifier to reduce the dimensions of my dataset. But right now, ...
1
vote
0answers
21 views

How to stack classifiers optimized for different score functions?

I have binary classification task (class0 vs class1) and I would like to create a stacked model out of classifiers which are individually optimized for different scorings. For example, let's say Clf_A ...
0
votes
1answer
14 views

Adding time as a feature with xgboost/random forests

I am trying to use xgboost for performing some regression and the features I have are rather simple and limited. I have the time stamp associated with some ...
2
votes
1answer
85 views

Feature Importance

I have a dataset with 10 features. I've computed the feature importance using permutation importance with cross-validation from eli5, after fitting an extremely randomized trees (ET) classifier form ...
0
votes
2answers
36 views

Model should predict the same value every time for the same input

I have used a random forest model for prediction of prices. Should the model be predictable in its behavior? By this, I mean that I'm not changing the model and the input, Will the predicted value ...
-1
votes
0answers
14 views

How do classification algorithms assign coefficients or importance to a categorical feature

I have a binary classification dataset with target variable being response (Y/N) and one of the predictor variable is month. Image has data on respondents count and rate by month (sorted by ...
1
vote
1answer
60 views

Can I arbitrarily eliminate 20% of my training data if doing so significantly improves model accuracy?

My dataset contains 2000 records with 125 meaningful fields 5 of which are distributed along highly skewed lognormal behavior. I've found that if I eliminate all records below some threshold of this ...
3
votes
2answers
48 views

Which classifier performs better when using 'class_weight'?

I have used the 'class_weight' method to balance my multi-class classification problem, using Logistic Regression, Random Forest, and XGBoost classifiers. Among these three methods, logistic ...
1
vote
0answers
75 views

Evaluating Random Forest regression model that predicts low values for skewed dependent variable

Background I'm trying to predict the value of website visitors. Only a small fraction of the visitors actually make a purchase, so ~97% of the visits has the value of 0, while about 2-3% has values ...
0
votes
1answer
29 views

Using a LinearSVC() for multilabel classification with MultiOutputClassifier() in a pipeline in sci-kit learn

My input data is a (23948,) pandas.Series of strings containing newspaper headlines. My target are 20 labels of the headline (e.g. 'crime', 'politics') each binarily encoded with [0, 1]. The labels ...
1
vote
1answer
23 views

How do we decide on the classification algorithm to use with huge training size?

I am solving a questions binary classification problem and the training size for this is huge(291 billion). The data has bloated because of using tfidfvectorizerfor ...
2
votes
2answers
56 views

Calculate future GDP % using machine learning

I need to estimate the GDP % of a country three years into the future, based on historic data. I have 30+ years of the following monthly data that includes features such as inflation and unemployment ...
2
votes
1answer
30 views

Force selecting samples in majority class with random forest

Context: I have some data to fit a random forest classifier (binary output) with 1 being a very rare event. In particular, in my training set, there are only 614 1's out of 29400 points. I am using ...
1
vote
1answer
55 views

Isolation forest - grouped by

I'm trying to use isolation forest algorithm for outliers detection. Data has 2 columns: id and REV. Below code gives me ...
0
votes
1answer
20 views

Is there a case that random forest data (a bunch of trees) consumes MB or GB of memory?

Is there any case that random forest (a bunch of trees) consumes too much memory in practice? I'm wondering why my scikit-learn consumes large amount of memory.
0
votes
0answers
11 views

What is the default method for Orange to deal with missing values in random forest classification?

Good day, I have built a random forest classification model in Orange, but some of my input data (all continuous) are missing. Up to 30% of the data for some variables are missing. I understand the ...
2
votes
1answer
22 views

Significant drop from validation accuracy to test accuracy

I am more familiar with classification tasks, though I have been working on a regression problem. I was given a large training dataset (>70k samples) and an independently collected test set (~2k). I ...
1
vote
2answers
39 views

ScikitLearn - RandomForestRegressor score different in and out of grid search

I am using RandomForestRegressor (scikit-learn python package). I am looking for the best values for hyperparameters ...
2
votes
2answers
88 views

Using random forest for selecting variables returns the entire dataframe

I am in the process of dimensionality reduction. I am using Random Forest to find the columns with the highest level of correlation with the target SalePrice column. The problem is that the output ...
2
votes
2answers
45 views

Decision Trees - how does split for categorical features happen?

A decision tree, while performing recursive binary splitting, selects an independent variable (say $X_j$) and a threshold (say $t$) such that the predictor space is split into regions {$X|X_j < t$} ...
1
vote
1answer
12 views

clarification on splitting individual trees in extra trees?

So I am a beginner in machine learning and just started learning about random trees in this article here. When it talks about tuning the hyperparameter K, I'm a bit confused as to how it works. It ...
1
vote
0answers
37 views

Choosing weights on random forest for imbalanced data with the aim to minimize false positives

I am currently dealing with a binary classification task on imbalanced data with the following distribution: ...
0
votes
1answer
20 views

How do I interpret the given classification report?

The given classification report was obtained from running a Random Forest binary classifier on the test data. There is huge class imbalance in the training data. How do I interpret the given ...
2
votes
1answer
49 views

Is it normal that a classifier always wrongly predicts the same samples?

I'm trying to improve the accuracy of a classifier, a random forest one. I built different models with the same hyperparameters but with different random seeds, trained them with the same training ...
2
votes
2answers
65 views

unimportant features impact on model's performance

Using XGBoost and RandomForests, do unimportant features (according to the feature_importances_ attribute) hurt the model's performance? Do I need to carefully ...
1
vote
1answer
40 views

Random Forest Prediction

Let's say I want to classify if the employee will churn or not. In my random forest, I have 6 estimators where 3 of them predict the employee to churn and the other estimators predict the employee to ...
-1
votes
2answers
25 views

How to approach a problem when the information is in the relationship between the points, and not the points itself?

I am trying to analyze vehicular mobility models, where I am trying to learn how a particular vehicle moves and then detect similar patterns from the testing data. Here's what I have done for now: I ...
0
votes
0answers
22 views

The Difference between One Hot Encoding and LabelEncoder? [duplicate]

I am working on a ML problem to predict house prices and Zip Code is one feature which will be useful. I am also trying to use ...
0
votes
2answers
37 views

Confusion on result of K-Fold Cross Validation and Independent Test set

I am relatively new in Machine Learning. I am using Random Forest and SVM for a project. Where I did a ...
0
votes
1answer
21 views

What does the KFold error mean and how to get confusion matrix from Kfold random forest implementation?

from sklearn.model_selection import KFold num_folds = 10 seed = 77 kf = KFold(n_splits=num_folds,random_state=77,shuffle=False) rfc=RandomForestClassifier(...
0
votes
0answers
13 views

How do I organize a multi-site multivariate time-series dataset for a Random Forest Regression?

I am trying to do a Random Forest Regression to forecast the next months value. I have a few years of data split by month. In each month I have about 1500 unique sites. There are 14 features.
1
vote
0answers
25 views

Alternatives for categorical prediction

Upfront question: What are some alternative methods for predicting categorical data? Details: I routinely process data that is 100% categorical. Almost always, this data is nominal (while, ...
1
vote
1answer
45 views

How to use Random Forest to reduce dimensions

I am working on the Boston competition on Kaggle and at the moment I am trying to use Random Forest to find the columns with the highest correlation with the target variable ...
0
votes
1answer
20 views

Getting a ValueError from train_test_split

I'm working on this dataset. I'm trying to select features using Random Forest. This is the relevant code: ...
1
vote
0answers
14 views

Scikit-learn randomforestclassifier error on fitting

So I am trying to use the random forest classifier from scikit-learn and I use tfidfvectorizer to create a feature set, then use test_train_split to create X_train and Y_train. I pass this into the ...
0
votes
0answers
20 views

Hyperopt Model runs with 0 seconds duration

I use Hyperopt for Random Forest Regression hyperparameter tuning. my parameterspace is : ...
1
vote
0answers
26 views

I have tried 5 different types of model but all returns really low training accuracy (~64%) and low testing accuracy (~14%). What should I do?

I am working with a typical regressor problem. There are $6$ features in the dataset that I am concerned with. There are about $800$ data points in my dataset. The features and the predicted values ...
0
votes
1answer
27 views

Random forest with zero precision for unbalanced test data

Apologies if this is a basic question. I have a very unbalanced dataset in which the records are labelled by one of two classes, class1 (negative class) and class2 (positive class): class 1: 1.5 ...
1
vote
0answers
28 views

Feature Importance Scores from Gradient Boosting vs Random Forest

In sklearn, the feature_importances_ attribute exists for both RandomForestClassifier and GradientBoostingClassifier. Would like to know what are the fundamental differences in how this attribute is ...
0
votes
1answer
13 views

How to create model for selecting a set of categories with a set of attributes?

I have a couple of hundred categories where each of these categories has a specific set of attributes having different values (historical). The problem I need to solve is to select the best set of ...
1
vote
1answer
31 views

Inputs required for Random Forest Regressor and ways to improve performance

I am using Random Forest Regressor to predict inventory needs. The data I am using to train the model lists the total quantity picked for each product per date, but does not include rows where total ...
1
vote
2answers
30 views

How to transform time series data to apply supervised learning algorithms to it?

Apologies in advance for what may be a very basic question. I have a dataset consisting of marketing calls to different clients, which include the timestamp for the call. My goal is to train a model ...
0
votes
2answers
65 views

RandomForest surprisingly high accuracy

I've been experimenting with Random Forests on Python after trying Naive Bayes which gave me lower accuracy than I expected, 62%. My csv file has around 14,000 records, I use 80% for the training set ...
1
vote
2answers
77 views

convert predict_proba results using class_weight in training

As my dataset is unbalanced(class 1: 5%, class 0: 95%) I have used class_weight="balanced" parameter to train a random forest classification model. In this way I penalize the misclassification of a ...
0
votes
0answers
8 views

Custom Imputation relative to targets in TRAIN and TEST sets

I have a methodology question for dealing with heaps of missing data in my project. My dataset is composed of parts A (~200 columns) and B (another ~200 columns). Together they are to be used for ...
2
votes
2answers
56 views

When to use Random Forest

I understand Random Forest models can be used both for classification and regression situations. Is there a more specific criteria to determine where a random forest model would perform better than ...
2
votes
1answer
88 views

Random forest classifier is predicting only one class even when the dataset is not imbalanced

This is a binary classification task, I have 15K 1's and 11K 0's (target) I have tried the following: ...