I am working on a text Categorization problem, the objective is to classify related companies into their corresponding categories. This is a single category classification problem and not multi-class classification. Below is the details of the method I used.
Data Preparation -
- Broke the documents in the list of tokens/ words.
- Removed numbers, stop words, punctuations.
- Performed stemming.
- Convert corpus into DTM and,
- specify the word length in-between(3, 10)
- Transformed the documents into TF-IDF vectors
- convert it into a matrix and further into a data frame
These will be used a feature list for the classification algorithm. Used Random forest ("ranger" by using mlr).
I have 853 categorized documents with 20 categories, and I used 75:25 ration for training and test dataset. I used mlr package in R. The classification accuracy I have managed to get is 45% and I need it to be at least 80%.
I also applied xgboost but got only 38% accuracy.
Here is an example code I used so far: -
terms <-DocumentTermMatrix(clean_corp, control=list(wordLengths=c(3, 9), weighting = weightTfIdf)) m <- as.matrix(terms) vertical_market <- process$label termsDf <- as.data.frame(m) termsDf$label <- as.factor(label) names(termsDf) = lapply(names(termsDf), as.name) words <- colnames(termsDf) # taking words with length more than 4 data <- termsDf[, str_length(words) < 16 & str_length(words) > 3] colnames(data) <- c(paste0("w", 1:(ncol(data)-1)), "label") task <- makeClassifTask(data = data, target = "label") lrn <- makeLearner("classif.ranger", mtry=100,respect.unordered.factors = FALSE, predict.type = "response") n <- nrow(data) set.seed(123) train.set <- sample(n, size = 2/3*n) test.set <- setdiff(1:n, train.set) model <- mlr::train(lrn, task, subset = train.set) pred <- predict(model, task = task, subset = test.set) performance(pred, measures = list(mmce,acc))
Any help on how I can improve the accuracy would be greatly appreciated. Thanks a lot. Please let me know if you need any more details.