4
$\begingroup$

I am trying to build a CART model using rpart on a data set with around 7k rows and 456 columns

cmodel2=rpart(DV ~ .,data=teltrain2,method="class")

This has never returned a result yet. I am using a 16GB ram machine with R 3.2.3. Every time on running this, R execution goes on and on, I have to kill the R GUI process to finally stop it.

Description of the columns

  • id --> unique id (which I have removed from the training set)

  • Location --> number

  • col3 to col400 --> these have been hot encoded by me and include just 0 or 1 as values

  • col400-col456 --> these are features which have numeric values like 23,144 etc, all less than 1000

  • DV --> dependent variable is categorical with more than 2 values.

I have tried running rpart with a smaller set of 1000 rows and 456 cols, still no luck. I don't want to do any kind of feature elimination on the first run as as of now I have no idea whether they affect the dependent variable or not.

I have tried mentioning minbucket=25,20,15,10 with no luck either.

My objective is to get the resulting model of rpart on the entire data set. What are my options?

$\endgroup$
  • $\begingroup$ are you still looking for an answer? $\endgroup$ – Toros91 Jan 26 '18 at 3:38
  • $\begingroup$ @Toros91 -I never got an answer on why rpart was giving such poor performance on not sol large dataset,so yes still looking for answer around performance of rpart.I however moved to randomForest and GBM and found solution to my original problem. $\endgroup$ – Bak_was Jan 29 '18 at 16:38
  • $\begingroup$ This question is about a problem with the r package, not about the performance of the algorithm. The answers below are not helpful $\endgroup$ – JanLauGe Jan 24 '19 at 11:10
1
$\begingroup$

Have you tried using randomForest instead of rpart? For example, let's assume you have two data.frames:

train_data and test_data

In my example, the last column is the class (and is a factor variable), and all other variables (all other columns) are numeric. For example, one training example might look like:

4 5 10 12 1 0

where 0 is the class label. You can do the following to run a random forest with 50 trees:

library(randomForest)
model_pl_1 <- randomForest(train_data[,-ncol(train_data)], train_data[,ncol(train_data)], test_data[,-ncol(test_data)], test_data[,ncol(test_data)], ntree=50)
| improve this answer | |
$\endgroup$
  • $\begingroup$ ok i will try rf as well ,but any idea why CART won't work but rf would ? $\endgroup$ – Bak_was May 31 '17 at 4:01
  • $\begingroup$ Sorry, no idea. I've just had better luck with randomForest than with rpart. I find rpart takes forever. Maybe it's doing more computation? $\endgroup$ – StatsSorceress May 31 '17 at 18:46
  • 1
    $\begingroup$ Random Forest uses a square root of the number of predictors for each tree. 21 vs over 400. $\endgroup$ – Chris Jun 6 '17 at 13:41
1
$\begingroup$

From experience, I've found that ranger provides a quick implementation of randomForest (probably since it is designed specifically to run on high-dimensional data sets). It's also more flexible than randomForest while providing comparable accuracy.

| improve this answer | |
$\endgroup$
1
$\begingroup$

Firstly, you need to know how RPART works, it is a decision tree which works by building a tree based on the features available i.e, Trees (also called decision trees, recursive partitioning) are a simple yet powerful tool in predictive statistics. The idea is to split the co-variable space into many partitions and to fit a constant model of the response variable in each partition. In case of regression, the mean of the response variable in one node would be assigned to this node.

The structure is similar to a real tree (from the bottom up): there is a root, where the first split happens. After each split, two new nodes are created (assuming we only make binary splits). Each node only contains a subset of the observations. The partitions of the data, which are not split any more, are called terminal nodes or leafs. This simple mechanism makes the interpretation of the model pretty easy. to explain it with an example.

Now coming to your scenario, as you have mentioned it has 400+ features, now it start with a target variable and then divides into 2 parts, and goes to feature 1 and divides based on the value present in it like 0 or 1(for example 0 on the left node and 1 on the right node) and then it goes on till the last feature(sometimes it might not go till last feature) like that literally 400+ levels are generated for each and every record(but not 400 everytime because some records are similar, so those records gets generalized). So now you imagine for 400+ features, 7000 records, how big the tree might become and how hard it would be to generalize the results. So RPART couldn't give any result.

Now, how did by using RF we achieved the results? RF works as Ensemble of trees i.e., it splits the data at row level and column level by which the tree generated are small and it takes ensemble of all the trees and gives an average(other ways too like voting etc) of all the small trees.

Go through these links for better understanding of RPART, Link-1

Go through these links for better understanding of RandomForest, Link-2.

My suggestion to get better results using Rpart is possible by reducing the number of features by applying PCA or any Dimensionality Reduction algorithms. Since you have obtained desired results using different algorithms so the above is for your understanding.

Do let me know if you have any additional questions.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Let me know if you have any doubts, you can accept the answer if you got what you are looking for. Thanks in advance! $\endgroup$ – Toros91 May 4 '18 at 7:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.