I try to implement in R test for appropriate metrics for lda.
Here the way I try to use LDA
require(quanteda) require(quanteda.corpora) require(lubridate) require(topicmodels) dtext <- data.frame(id = c(1,2,3), text = c("This dataset contains movie reviews along with their associated binary sentiment polarity labels. It is intended to serve as a benchmark for sentiment classification. This document outlines how the dataset was gathered, and how to use the files provided.", "The core dataset contains 50,000 reviews split evenly into 25k train and 25k test sets. The overall distribution of labels is balanced (25k pos and 25k neg). We also include an additional 50,000 unlabeled documents for unsupervised learning.", "There are two top-level directories [train/, test/] corresponding to the training and test sets. Each contains [pos/, neg/] directories for the reviews with binary labels positive and negative. Within these directories, reviews are stored in text files named following the convention [[id]_[rating].txt] where [id] is a unique id and [rating] is the star rating for that review on a 1-10 scale. For example, the file [test/pos/200_8.txt] is the text for a positive-labeled test set example with unique id 200 and star rating 8/10 from IMDb. The [train/unsup/] directory has 0 for all ratings because the ratings are omitted for this portion of the dataset."),stringsAsFactors = F) corp_news <- corpus(dtext) dfmat_news <- dfm(corp_news, remove_punct = TRUE, remove = stopwords('en')) %>% dfm_remove(c('*-time', '*-timeUpdated', 'GMT', 'BST')) %>% dfm_trim(min_termfreq = 0.95, termfreq_type = "quantile", max_docfreq = 0.1, docfreq_type = "prop") dfmat_news <- dfmat_news[ntoken(dfmat_news) > 0,] dtm <- convert(dfmat_news, to = "topicmodels") lda <- LDA(dtm, k = 10)
Which are the parameters which I have to test to make the perfomance of lda better? example there methods for the topic selection number algorithms like gibbs, recover and recoverl2 what other tunning it needs?