1
$\begingroup$

I am using Boruta feature selection (with Random forest) to decide the important features in the below data set.

   Gender Married Dependents     Education Self_Employed  ApplicantIncome  \
0   Male      No          0      Graduate            No             5849   
1   Male     Yes          1      Graduate            No             4583   
2   Male     Yes          0      Graduate           Yes             3000   
3   Male     Yes          0  Not Graduate            No             2583   
4   Male      No          0      Graduate            No             6000   

 CoapplicantIncome  LoanAmount  Loan_Amount_Term  Credit_History  \
0                0.0         NaN             360.0             1.0   
1             1508.0       128.0             360.0             1.0   
2                0.0        66.0             360.0             1.0   
3             2358.0       120.0             360.0             1.0   
4                0.0       141.0             360.0             1.0   

Property_Area Loan_Status  
0         Urban           Y  
1         Rural           N  
2         Urban           Y  
3         Urban           Y  
4         Urban           Y

Please help me in clarifying the below doubts 1) whether I need to convert the all categorical variable into numeric variable (using one hot encoding) before applying Boruta? 2) Whether the NA values would be taken care by Boruta or do we need to remove NA values before feeding into Boruta ?

In case of Regression problem, whether the Boruta approach (Random forest classifier -> Boruta) remains the same as in classification problem ?

Thank you.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

In R, Boruta relies on the ranger implementation of random forest. So:

  1. Converting input variables from categorical to numeric is not necessary.
  2. You will need to address NA values prior to running the algorithm.

Be aware that Boruta can be very slow!

Improve this answer
$\endgroup$
4
  • $\begingroup$ thanks for the reply. How about in Python ? Do I need to take care of Categorical variables ? What are the other simpler and faster method to perform feature selection ? $\endgroup$
    – deepguy
    May 2, 2018 at 12:34
  • $\begingroup$ I'm not sure about Python - my background is in R. In terms of other techniques, sometimes ANOVA / F-tests / Chi-squared / Cramer's V yield good results. You can always replicate the Boruta logic in another ML framework... for example, use boosted tree models instead of random forests. $\endgroup$
    – bradS
    May 3, 2018 at 8:37
  • $\begingroup$ Thanks for that information. In case of Regression problem, whether the Boruta approach (Random forest classifier -> Boruta) remains the same as in classification problem ? $\endgroup$
    – deepguy
    May 10, 2018 at 19:52
  • $\begingroup$ The underlying logic is the same for both regression and classification. The underlying random forests will either be composed of regression or classification trees depending on the type of response variable. $\endgroup$
    – bradS
    May 11, 2018 at 8:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.