I'm doing credit risk modelling and the data have large number of features.I am using boruta package for feature selection. The package is too computationally expensive, I cannot run it on the complete training dataset. What i'm trying to do is take a subset of the training data(let's say about 20-30%) and run the boruta package on that subsetted data and get the important features. But when i use random forest to train the data I have too use the full dataset. My question is, Is it right to select features only on a part of train data but then build the model on whole of training data?
$\begingroup$
$\endgroup$
3
-
$\begingroup$ I think it's a valid question. The OP is asking whether it's OK to use a subset of her/his data to perform feature selection - the question is pretty clear to me. BTW, please clarify how many features you have, and how large is your sample. The answer depends on these numbers. $\endgroup$– famargarCommented May 3, 2017 at 10:39
-
$\begingroup$ I have 18 features. Dataset is about 150mb. I'm using R studio on a 8gb ram,6th gen i5 processor. $\endgroup$– Dhruv MahajanCommented May 3, 2017 at 14:53
-
$\begingroup$ Thanks. You can certainly do feature selection on 10-30% of your data - with your numbers I am assuming that would still amounts to tens of thousands of rows of data, more than enough to reliably do feature selection. I am not familiar with Boruta, my answer is driven by basic statistics. $\endgroup$– famargarCommented May 3, 2017 at 15:45
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Technically, it will be alright to use a sample of the data. One important assumption is that most of the features don't have outlier. In such case your sampling might miss out on those.
But, I don't see a necessity of sampling with the size of data that you have given. 150mb is not large data at all, given that you have 8 GB ram at disposal. Rather than sampling the data, first check if there is alternate way of selecting the features. Since there are only 18 columns, look at the summaries for each columns, how do the features correlate with your outcome variable, which variables add very little information. If you have not done more exploratory analysis of the 18 variables, I'll suggest you do that first.
Secondly, why not build your random forest directly with 18 features? If you had, say, 180 features, it would have made sense to do automated feature selection first using Boruta etc. If there are features that do not add information, their variable importance in forest model will be low. You can drop them in subsequent analysis
