Recently I've been doing some text mining and the stemming processing is taking a lot of time (I'm working with portuguese language).

I found the hunspell package, that provides spell checking, corrections and stemming.

Also, a friend developed another stemming function that I've been studying (RSLP: Removedor de Sufixos da Língua Portuguesa).

But both were made to work on a char vector, and I need to make them work with the tm_map function that read a corpus.

I know that to make them work, I need to split the documents on the corpus into single words, but I don't know how.

  • 1
    $\begingroup$ Hi, I can't test because I am using an old verion of tm (upgrading would break my code). Basically, you wan't to use your own custom function within tm_map. For that you need to encapsulate it within content_transformer. Cf for instance rstudio-pubs-static.s3.amazonaws.com/… and tm_transformerhelp. Now your function should be something like mytxtfoo <-content_transformer(function(txt){paste(mystemfunction(strsplit(txt, " ")[[1]]))}) $\endgroup$ – Eric Lecoutre Jul 5 '16 at 9:41
  • $\begingroup$ Hey Eric, thanks for your answer, it seems to be exactly what I was looking for. Now I just need some time to fully understand the RSLP function that I'm working with to try your suggestion. $\endgroup$ – TheBiro Jul 5 '16 at 18:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.