Questions tagged [ngrams]
The ngrams tag has no usage guidance.
34
questions
1
vote
1
answer
75
views
Which should we choose: sequence-model vs n-gram model and why does it depend on ratio of Samples / Words per Sample
This ML tutorial from Google is analyzing the imdb reviews dataset to predict the tag positive or negative. When choosing a model
Calculate the number of samples/number of words per sample ratio.
If ...
- 423
3
votes
1
answer
353
views
FastText Model Explained
I was reading the FastText paper and I have a few questions about the model used for classification. Since I am not from NLP background, some I am unfamiliar with the jargon.
In the figure, what ...
CommunityBot
- 1
4
votes
1
answer
470
views
In smoothing of n-gram model in NLP, why don't we consider start and end of sentence tokens?
When learning Add-1 smoothing, I found that somehow we are adding 1 to each word in our vocabulary, but not considering start-of-sentence and end-of-sentence as two words in the vocabulary. Let me ...
CommunityBot
- 1
1
vote
0
answers
40
views
How does FastText create n-gram word features?
In the paper Bag of Tricks for Efficient Text Classification they talk about creating n-gram (word) features, and in their experiments they show results for both 1-gram and bi-gram.
As far as I ...
- 500
1
vote
0
answers
108
views
What does NIST information weights refer to?
NIST is a metric used to measure the goodness of translation.
In the paper, Doddington (2002) introduce the notion of "Information weights"
Information weights were computed using N-gram ...
- 289
1
vote
1
answer
455
views
Is n-gram a special instance of bag of word? What are their differences?
Is n-gram a special instance of bag of word? What are their differences? From my understanding, n-gram is when replacing the words in bag of words with n-grams, and follow the same procedures to ...
- 26.6k
2
votes
1
answer
48
views
Classifying short strings of text with additional context
I have a list of short strings each identifying a city. Misspellings are very common. The example below shows some of these short strings, along with the correct city they're supposed to match.
...
- 21.2k
1
vote
1
answer
91
views
N-gram language model for preposition predition
I am trying to build N gram models to predict the missing prepositions of a text corpus.
I would want to have some guidance on if I'm understanding and doing things correctly.
So the N gram model is ...
- 25.3k
1
vote
0
answers
59
views
Does N-gram language model for text generation are more efficient than Neural Network language models?
I recently build an language model with N-gram model for text generation and for change I started exploring Neural Network for text generation. One thing I observed that the previous model results ...
- 11
1
vote
1
answer
32
views
How to keep only the top k-frequent ngrams in a text field with pandas?
How to keep only the top k-frequent ngrams in a text field with pandas? For example, I've a text column. For every row in it, I only want to keep those substrings that belong to the top k-frequent ...
- 6,568
1
vote
1
answer
53
views
Application of bag-of-ngrams in feature engineering of texts
I've got few questions about the application of bag-of-ngrams in feature engineering of texts:
How to (or can we?) perform word2vec on bag-of-ngrams?
As the feature space of bag of n-gram increases ...
- 6,568
3
votes
0
answers
58
views
Understanding Kneser-Ney Formula for implementation
I am trying to implement this formula in Python
$$ \frac{\text{max}(c_{KN}(w^{i}_{i-n+1} - d), 0)}{c_{KN}(w^{i-1}_{i-n+1})} + \lambda(c_{KN}(w^{i-1}_{i-n+1})\mathbb{P}(c_{KN}(w_{i}|w^{i-1}_{i-n+2})$$
...
- 237
1
vote
1
answer
32
views
N-Gram Smoothing
I am wondering if there is a good example out there that compares N-Gram with various smoothing techniques.
I found this notebook that applies Laplace transform but that is about it. Any suggestions ...
- 25.3k
1
vote
2
answers
720
views
How do I get ngrams for all combinations of words in a sentence?
Lets say I have a sentence "I need multiple ngrams". If I create bigrams using Tf idf vectorizer it will create bigrams only using consecutive words. i.e. I will get "I need", &...
- 401
2
votes
1
answer
46
views
Usage of KL divergence to improve BOW model
For a university project, I chose to do sentiment analysis on a Google Play store reviews dataset. I obtained decent results classifying the data using the bag of words (BOW) model and an ADALINE ...
- 1,633
4
votes
1
answer
42
views
Artificially increasing frequency weight of word ending characters in word building
I have a database of letter pair bigrams. For example:
...
CommunityBot
- 1
6
votes
2
answers
329
views
N-grams for RNNs
Given a word $w_{n}$ a statistical model such a Markov chain using n-grams predicts the subsequent word $w_{n+1}$. The prediction is by no means random.
How is this translated into a neural model? I ...
- 314
2
votes
1
answer
176
views
NLP: find the best preposition for connecting parts of a sentence
My task is to connect 2-3 parts of the sentence into one whole using a preposition
the first part is some kind of action. Ex. "take pictures"
the second part is an object that can ...
- 25.3k
12
votes
1
answer
416
views
ngram and RNN prediction rate wrt word index
I tried to plot the rate of correct predictions (for the top 1 shortlist) with relation to the word's position in sentence :
I was expecting to see a plateau sooner on the ngram setup since it ...
CommunityBot
- 1
1
vote
1
answer
142
views
what is the training phase in N-gram model?
Following is my understanding of N gram model used in text prediction case :
Given a sentence say, " I love my " (say N = 1 /bigram), using N gram and say 4 possible candidates ( country, ...
- 796
0
votes
0
answers
116
views
Shouldn't ROUGE-1 precision be equal to BLEU with w=(1, 0, 0, 0) when brevity penalty is 1?
I am trying to evaluate a NLP model using BLEU and ROUGE. However, I am a bit confused about the difference between those scores. While I am aware that ROUGE is aimed at recall whilst BLEU measures ...
- 26.6k
1
vote
0
answers
37
views
Self Organising Map with variable length ordered sets of N-grams
I want to preface my question with the highlighted situation I have might not be applicable to kohonen self organising maps (SOM) due to a lack of understanding on my part so I do apologise if that is ...
- 111
1
vote
0
answers
64
views
For an n-Gram model with n>2, do we need more context at end of each sentence?
Jurafsky's book says we need to add context to left and right of a sentence:
Does this mean,
for example, if we've a corpus of three sentences: ...
- 123
0
votes
1
answer
61
views
Why is n-grams language independent?
I don't understand how n-grams are language independent. I've read that by using character n-grams of a word than the word itself as dimensions of a vector space model, we can skip the language-...
- 1,364
6
votes
2
answers
5k
views
Clustering or classifing n-gram-based text categories
I have large set of data records looking like this:
"text", "category"
I extract n-grams from text (2-, 3- and 4-grams) and store count of each n-gram per ...
CommunityBot
- 1
2
votes
1
answer
3k
views
1
vote
1
answer
159
views
Is the search for a specific n-gram the same like a string search?
Is the result of a search for a specific n-gram like sherlock+holmes equal to the result of a regex search for "sherlock holmes" in the same document corpus?
So if i read about n-grams for certain ...
- 7,456
1
vote
1
answer
569
views
How to feed data for ngram model?
I want to train an ngram language model
Let's say I have the following corpus:
...
- 25.3k
1
vote
0
answers
53
views
Dealing with missing n-grams in Naive Bayes classifier
I am doing sentiment analysis on code-mixed text data, i.e English used interchangeably with another language. The dataset I currently have is very small in size, approx 3.5k samples.
I am sure that ...
- 21
1
vote
0
answers
111
views
Skip-thought models applied to phrases instead of sentences
My goal is to build a statistical model with domain specific phrase embeddings.
To do this, I am doing research on how to build a model using skip-thought vectors, where instead of using sentence ...
- 201
0
votes
2
answers
240
views
Discarding rare words when comparing texts - per text, per comparison, or per codex?
I'm trying to compare texts (read: books) using KL divergence of N-gram usage frequency.
first I have to calculate the frequency of N-grams, and I see (perhaps unsurprisingly) that many of the words ...
- 196
2
votes
1
answer
4k
views
How to improve Naive Bayes?
I am solving a problem that address this question "What are the Actions that lead to high or low score?"
I have the following Data that consist of text and score , I want to derive the words or ...
- 2,392
1
vote
1
answer
816
views
What affect will replacing words with bigrams have on TfIDF?
Say I have a corpus of text documents on which I have calculated each documents TfIDF vector. With this sparse matrix representation of the corpus, I can calculate similarities between documents by ...
CommunityBot
- 1
1
vote
1
answer
41
views
Predicting interactions [closed]
First off, I don't really know much about machine learning.
In a virtual world, such as a video game like minecraft or an application like Google Street view, a user can navigate the world using the ...
- 499