# How to improve the accuracy of Random Forest for Text Categorization

I am working on a text Categorization problem, the objective is to classify related companies into their corresponding categories. This is a single category classification problem and not multi-class classification. Below is the details of the method I used.

Data Preparation -

• Broke the documents in the list of tokens/ words.
• Removed numbers, stop words, punctuations.
• Performed stemming.
• Convert corpus into DTM and,
• specify the word length in-between(3, 10)
• Transformed the documents into TF-IDF vectors
• convert it into a matrix and further into a data frame

These will be used a feature list for the classification algorithm. Used Random forest ("ranger" by using mlr).

I have 853 categorized documents with 20 categories, and I used 75:25 ration for training and test dataset. I used mlr package in R. The classification accuracy I have managed to get is 45% and I need it to be at least 80%.

I also applied xgboost but got only 38% accuracy.

Data Looked like the below-mentioned image: -

Here is an example code I used so far: -

terms <-DocumentTermMatrix(clean_corp,
control=list(wordLengths=c(3, 9),
weighting = weightTfIdf))

m <- as.matrix(terms)
vertical_market <- process$label termsDf <- as.data.frame(m) termsDf$label <- as.factor(label)
names(termsDf) = lapply(names(termsDf), as.name)

words <- colnames(termsDf)

# taking words with length more than 4
data <- termsDf[, str_length(words) < 16 & str_length(words) > 3]
colnames(data) <- c(paste0("w", 1:(ncol(data)-1)), "label")

lrn <- makeLearner("classif.ranger", mtry=100,respect.unordered.factors = FALSE,
predict.type = "response")
n <- nrow(data)
set.seed(123)
train.set <- sample(n, size = 2/3*n)
test.set <- setdiff(1:n, train.set)
model <- mlr::train(lrn, task, subset = train.set)