How to improve accuracy further for forest cover prediction

Question

I am doing Forest-Cover-Type-prediction on Kaggle

this is the train and test data

> dim(train)
[1] 15120    56
> dim(test)
[1] 565892     56

So far I have done this :

combined this data comb <- rbind(train,test)

then mapped 40 mutually exclusive Soil_Type columns into one and 4 mutually exclusive Wilderness_Area column into one.

comb$Soil <- apply(comb[grep("Soil_Ty+",colnames(comb))], 1 , function(x){which(x == 1)})
comb$Wilderness <- apply(comb[grep("Wilderness_Are+",colnames(comb))], 1 , function(x){which(x == 1)})

after deleting these 44 columns I got it down to 14 columns.

> dim(comb)
[1] 581012     14
> colnames(comb)
 [1] "Id"                                 "Elevation"                          "Aspect"                            
 [4] "Slope"                              "Horizontal_Distance_To_Hydrology"   "Vertical_Distance_To_Hydrology"    
 [7] "Horizontal_Distance_To_Roadways"    "Hillshade_9am"                      "Hillshade_Noon"                    
[10] "Hillshade_3pm"                      "Horizontal_Distance_To_Fire_Points" "Cover_Type"                        
[13] "Soil"                               "Wilderness"       

then I separated the data

train <- comb[1:15120,]
test <- comb[15121:581012,]

and ran randomForest on this data

set.seed(415)
fit <- randomForest(Cover_Type ~ .,data=train[-1], importance=TRUE, ntree=2000, na.action = na.omit)
varImpPlot(fit)
predi <- predict(fit,test)

and got an accuracy of 0.70372 on Kaggle. Now I have hit the wall. I tried plotting various variables against the Cover_Type(variable to be predicted) but couldn't do figure out what to do with those.

How do I improve accuracy from here? what is the general approach in such cases?

Answer 1

Couple of suggestions: Your training set is smaller than your test set? It should be the other way around.

You should also tune the RF hypermeters using the held out set or Cross Validation. The two parameter people tune on RF are ntrees and mtry.

Answer 2

You can try random search or grid search for better accuracy. Also you can create new models using variable importance.

http://machinelearningmastery.com/tune-machine-learning-algorithms-in-r