1
$\begingroup$

I am using Boruta feature selection (with Random forest) to decide the important features in the below data set.

   Gender Married Dependents     Education Self_Employed  ApplicantIncome  \
0   Male      No          0      Graduate            No             5849   
1   Male     Yes          1      Graduate            No             4583   
2   Male     Yes          0      Graduate           Yes             3000   
3   Male     Yes          0  Not Graduate            No             2583   
4   Male      No          0      Graduate            No             6000   

 CoapplicantIncome  LoanAmount  Loan_Amount_Term  Credit_History  \
0                0.0         NaN             360.0             1.0   
1             1508.0       128.0             360.0             1.0   
2                0.0        66.0             360.0             1.0   
3             2358.0       120.0             360.0             1.0   
4                0.0       141.0             360.0             1.0   

Property_Area Loan_Status  
0         Urban           Y  
1         Rural           N  
2         Urban           Y  
3         Urban           Y  
4         Urban           Y  

Please help me in clarifying the below doubts 1) whether I need to convert the all categorical variable into numeric variable (using one hot encoding) before applying Boruta? 2) Whether the NA values would be taken care by Boruta or do we need to remove NA values before feeding into Boruta ?

In case of Regression problem, whether the Boruta approach (Random forest classifier -> Boruta) remains the same as in classification problem ?

Thank you.

$\endgroup$
3
$\begingroup$

In R, Boruta relies on the ranger implementation of random forest. So:

  1. Converting input variables from categorical to numeric is not necessary.
  2. You will need to address NA values prior to running the algorithm.

Be aware that Boruta can be very slow!

$\endgroup$
  • $\begingroup$ thanks for the reply. How about in Python ? Do I need to take care of Categorical variables ? What are the other simpler and faster method to perform feature selection ? $\endgroup$ – deepguy May 2 '18 at 12:34
  • $\begingroup$ I'm not sure about Python - my background is in R. In terms of other techniques, sometimes ANOVA / F-tests / Chi-squared / Cramer's V yield good results. You can always replicate the Boruta logic in another ML framework... for example, use boosted tree models instead of random forests. $\endgroup$ – bradS May 3 '18 at 8:37
  • $\begingroup$ Thanks for that information. In case of Regression problem, whether the Boruta approach (Random forest classifier -> Boruta) remains the same as in classification problem ? $\endgroup$ – deepguy May 10 '18 at 19:52
  • $\begingroup$ The underlying logic is the same for both regression and classification. The underlying random forests will either be composed of regression or classification trees depending on the type of response variable. $\endgroup$ – bradS May 11 '18 at 8:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.