I am working on a text Categorization problem, the objective is to classify related companies into their corresponding categories. This is a single category classification problem and not multi-class classification. Below is the details of the method I used.

Data Preparation -

  • Broke the documents in the list of tokens/ words.
  • Removed numbers, stop words, punctuations.
  • Performed stemming.
  • Convert corpus into DTM and,
  • specify the word length in-between(3, 10)
  • Transformed the documents into TF-IDF vectors
  • convert it into a matrix and further into a data frame

These will be used a feature list for the classification algorithm. Used Random forest ("ranger" by using mlr).

I have 853 categorized documents with 20 categories, and I used 75:25 ration for training and test dataset. I used mlr package in R. The classification accuracy I have managed to get is 45% and I need it to be at least 80%.

I also applied xgboost but got only 38% accuracy.

Data Looked like the below-mentioned image: - Click to see the image of data

Here is an example code I used so far: -

terms <-DocumentTermMatrix(clean_corp,
                                      control=list(wordLengths=c(3, 9), 
                                                   weighting = weightTfIdf))

m <- as.matrix(terms)
vertical_market <- process$label
termsDf <- as.data.frame(m)
termsDf$label <- as.factor(label)
names(termsDf) = lapply(names(termsDf), as.name)

words <- colnames(termsDf)

# taking words with length more than 4
data <- termsDf[, str_length(words) < 16 & str_length(words) > 3]
colnames(data) <- c(paste0("w", 1:(ncol(data)-1)), "label")

    task <- makeClassifTask(data = data, target = "label")
    lrn <- makeLearner("classif.ranger", mtry=100,respect.unordered.factors = FALSE, 
                       predict.type = "response")
    n <- nrow(data)
    train.set <- sample(n, size = 2/3*n)
    test.set <- setdiff(1:n, train.set)
    model <- mlr::train(lrn, task, subset = train.set)
    pred <- predict(model, task = task, subset = test.set)
    performance(pred, measures = list(mmce,acc))

Any help on how I can improve the accuracy would be greatly appreciated. Thanks a lot. Please let me know if you need any more details.

  • $\begingroup$ Can we see some samples of the data? $\endgroup$ – JahKnows Jan 23 '18 at 9:05
  • $\begingroup$ @JahKnows, i edited my question. Now you can see the image of sample data. $\endgroup$ – Ahsan Nawaz Jan 23 '18 at 12:16
  • $\begingroup$ Show the examples of wrong classified texts and some train texts with same labels. May be you have no initially or dropped while cleaning data some info without which you can't put labels right on the test data, $\endgroup$ – CrazyElf Jan 23 '18 at 14:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.