Questions tagged [lsi]
The lsi tag has no usage guidance.
7
questions
3
votes
1
answer
123
views
Tokenizing words of length 1, what would happen if I do topic modeling?
Suppose my dataset contains some very small documents (about 20 words each). And each of them may have words in at least two languages (combination of malay and english, for instance). Also there are ...
- 241
2
votes
2
answers
112
views
How to represent a document in test data with the Document-Term Matrix created from the training set?
I build a classifier of documents using the vector representation of each document in the training set (i.e a row in the Document-Term Matrix). Now I need to test the model on the test data. But how ...
- 145
2
votes
1
answer
33
views
Topic modelling on only 24 documents gives the same "topic" for any K
Description:
I have 24 documents, each one of around 2.5K tokens. They are public speeches.
My text preprocessing pipeline is a generic one, including punctuation removal, expansion of English ...
- 31
2
votes
0
answers
135
views
Latent Semantic Indexing False Positive Detection
I am using the Gensim LsiModel. I have a set of documents and fixed set of topics. Some of the documents are already categorized, others are not. The goal is to categorize the uncategorized documents ...
user13684
1
vote
0
answers
29
views
document similarity when the document size less than 30 tokens?
I was solving a problem to compare 3 million, 2018 documents against the 2019 documents. There are three text attributes to be compared from one item against the other. I used Latent Semantic Indexing ...
- 111
1
vote
0
answers
166
views
What is "energy spectrum" in Latent Semantic Indexing (LSI)?
What is meant by energy spectrum in LSI(Latent Semantic Indexing)?
I am doing topic modeling with gensim's LsiModel, and part of the output per chunk is the following:
...
0
votes
0
answers
33
views
LSA Model Improvement
I followed gensim's Core Tutorial and build an LSA Classification, topic modeling and Document Similarity model for newsgroups dataset.
My code is available here.
I need help with below 3 concepts.
...
- 1