Recently I've been doing some text mining and the stemming processing is taking a lot of time (I'm working with portuguese language).
I found the hunspell
package, that provides spell checking, corrections and stemming.
Also, a friend developed another stemming function that I've been studying (RSLP: Removedor de Sufixos da Língua Portuguesa).
But both were made to work on a char vector, and I need to make them work with the tm_map
function that read a corpus.
I know that to make them work, I need to split the documents on the corpus into single words, but I don't know how.
tm
(upgrading would break my code). Basically, you wan't to use your own custom function withintm_map
. For that you need to encapsulate it withincontent_transformer
. Cf for instance rstudio-pubs-static.s3.amazonaws.com/… andtm_transformer
help. Now your function should be something likemytxtfoo <-content_transformer(function(txt){paste(mystemfunction(strsplit(txt, " ")[[1]]))})
$\endgroup$ – Eric Lecoutre Jul 5 '16 at 9:41