# Tag Info

49

There's a number of different ways of going about this depending on exactly how much semantic information you want to retain and how easy your documents are to tokenize (html documents would probably be pretty difficult to tokenize, but you could conceivably do something with tags and context.) Some of them have been mentioned by ffriend, and the paragraph ...

40

HDP is an extension of LDA, designed to address the case where the number of mixture components (the number of "topics" in document-modeling terms) is not known a priori. So that's the reason why there's a difference. Using LDA for document modeling, one treats each "topic" as a distribution of words in some known vocabulary. For each document a mixture ...

35

For example, for word $w$ at position $pos \in [0, L-1]$ in the input sequence $\boldsymbol{w}=(w_0,\cdots, w_{L-1})$, with 4-dimensional embedding $e_{w}$, and $d_{model}=4$, the operation would be \begin{align*}e_{w}' &= e_{w} + \left[sin\left(\frac{pos}{10000^{0}}\right), cos\left(\frac{pos}{10000^{0}}\right),sin\left(\frac{pos}{10000^{2/4}}\right),...

31

Shallow Natural Language Processing technique can be used to extract concepts from sentence. ------------------------------------------- Shallow NLP technique steps: 1) Convert the sentence to lowercase 2) Remove stopwords (these are common words found in a language. Words like for, very, and, of, are, etc, are common stop words) 3) Extract n-gram i.e., ...

29

Your problem can be solved with Word2vec as well as Doc2vec. Doc2vec would give better results because it takes sentences into account while training the model. Doc2vec solution You can train your doc2vec model following this link. You may want to perform some pre-processing steps like removing all stop words (words like "the", "an", etc. that don't add ...

27

Why to use deep networks? Let's first try to solve very simple classification task. Say, you moderate a web forum which is sometimes flooded with spam messages. These messages are easily identifiable - most often they contain specific words like "buy", "porn", etc. and a URL to outer resources. You want to create filter that will alert you about such ...

25

Anecdotally, I've never been impressed with the output from hierarchical LDA. It just doesn't seem to find an optimal level of granularity for choosing the number of topics. I've gotten much better results by running a few iterations of regular LDA, manually inspecting the topics it produced, deciding whether to increase or decrease the number of topics, and ...

23

What you're describing is often achieved using a simple combination of TF-IDF and extractive summarization. In a nutshell, TF-IDF tells you the relative importance of each word in each document, in comparison to the rest of your corpus. At this point, you have a score for each word in each document approximating its "importance." Then you can use these ...

22

Here's the link for the methods available for fasttext implementation in gensim fasttext.py from gensim.models.wrappers import FastText model = FastText.load_fasttext_format('wiki.simple') print(model.most_similar('teacher')) # Output = [('headteacher', 0.8075869083404541), ('schoolteacher', 0.7955552339553833), ('teachers', 0.733420729637146), ('teaches',...

22

Well the names are pretty straight-forward and should give you a clear idea of vector representations. The Word2Vec Algorithm builds distributed semantic representation of words. There are two main approaches to training, Distributed Bag of Words and The skip gram model. One involves predicting the context words using a centre word, while the other ...

21

Stop words are usually thought of as "the most common words in a language". However, other definitions based on different tasks are possible. It clearly makes sense to consider 'not' as a stop word if your task is based on word frequencies (e.g. tf–idf analysis for document classification). If you're concerned with the context (e.g. sentiment analysis) ...

20

The original paper "BLEU: a Method for Automatic Evaluation of Machine Translation" contains a couple of numbers on this: The BLEU metric ranges from 0 to 1. Few translations will attain a score of 1 unless they are identical to a reference translation. For this reason, even a human translator will not necessarily score 1. It is important to note that ...

19

I wanted to point out, since this is one of the top Google hits for this topic, that Latent Dirichlet Allocation (LDA), Hierarchical Dirichlet Processes (HDP), and hierarchical Latent Dirichlet Allocation (hLDA) are all distinct models. LDA models documents as dirichlet mixtures of a fixed number of topics- chosen as a parameter of the model by the user- ...

19

If you want to tackle the problem from another perspective, with an end to end learning, such that you don't specify ahead of time this large pipeline you've mentioned earlier, all you care about is the mapping between sentences and their corresponding SQL queries. Tutorials: How to talk to your database Papers: Seq2SQL: Generating Structured ...

18

Thank Abhishek. I've figure it out! Here are my experiments. 1). we plot a easy example: from gensim.models import Word2Vec from sklearn.decomposition import PCA from matplotlib import pyplot # define training data sentences = [['this', 'is', 'the', 'first', 'sentence', 'for', 'word2vec'], ['this', 'is', 'the', 'second', 'sentence'], ...

16

Here is what I learnt recently. Obviously, when talking about text generation RNNs we are talking about RNN language models. When asking about word/char-based text generation RNNs, we are asking about word/char-based RNN language models (LM). Word-based LMs display higher accuracy and lower computational cost than char-based LMs. This drop of ...

16

In short, there is nothing special about number of dimensions for convolution. Any dimensionality of convolution could be considered, if it fit a problem. The number of dimensions is a property of the problem being solved. For example, 1D for audio signals, 2D for images, 3D for movies . . . Ignoring number of dimensions briefly, the following can be ...

16

Cosine Similarity for Vector Space could be you answer. Or you could calculate the eigenvector of each sentences. But the Problem is, what is similarity? "This is a tree", "This is not a tree" If you want to check the semantic meaning of the sentence you will need a wordvector dataset. With the wordvector dataset you will able to check ...

15

The word2vec algorithm may be a good way to retrieve more elements for a list of similar words. It is an unsupervised "deep learning" algorithm that has previously been demonstrated with Wikipedia-based training data (helper scripts are provided on the Google code page). There are currently C and Python implementations. This tutorial by Radim Řehůřek, the ...

15

Personally I would advocate using something that is both not-specific to the NLP field, and something that is sufficiently general that it can still be used as a tool even when you've started moving beyond this level of metadata. I would especially pick a format that can be used regardless of development environment and one that can keep some basic structure ...

15

The answer to this question is that it depends. The primary approach is to pass in the tokenized sentences (so SentenceCorpus in your example), but depending on what your goal is and what the corpus is you're looking at, you might want to instead use the entire article to learn the embeddings. This is something you might not know ahead of time -- so you'll ...

15

Gazetteer or any other option of intentionally fixed size feature seems a very popular approach in academic papers, when you have a problem of finite size, for example NER in a fixed corpora, or POS tagging or anything else. I would not consider it cheating unless the only feature you will be using is Gazetteer matching. However, when you train any kind of ...

15

The closest would be like Jan has mentioned inhis answer, the Levenstein's distance (also popularly called the edit distance). In information theory and computer science, the Levenshtein distance is a string metric for measuring the difference between two sequences. Informally, the Levenshtein distance between two words is the minimum number of ...

15

When to use cosine similarity over Euclidean similarity Cosine similarity looks at the angle between two vectors, euclidian similarity at the distance between two points. Let's say you are in an e-commerce setting and you want to compare users for product recommendations: User 1 bought 1x eggs, 1x flour and 1x sugar. User 2 bought 100x eggs, 100x flour ...

14

Check out this link. Here, they will take you through loading unstructured text to creating a wordcloud. You can adapt this strategy and instead of creating a wordcloud, you can create a frequency matrix of terms used. The idea is to take the unstructured text and structure it somehow. You change everything to lowercase (or uppercase), remove stop words, ...

14

You must look at this Multicore implementation of t-SNE. I actually tried it and can vouch for its superior performance.

13

Here are the basic Natural Language Processing capabilities (or annotators) that are usually necessary to extract language units from textual data for sake of search and other applications: Sentence breaker - to split text (usually, text paragraphs) to sentences. Even in English it can be hard for some cases like "Mr. and Mrs. Brown stay in room no. 20." ...

13

During NLP and text analytics, several varieties of features can be extracted from a document of words to use for predictive modeling. These include the following. ngrams Take a random sample of words from words.txt. For each word in sample, extract every possible bi-gram of letters. For example, the word strength consists of these bi-grams: {st, tr, re, ...

13

Word embeddings are trained by substitutability, not similarity. If you consider a sentence like "This food is unflavored." Then a good substitute word would be "flavored", and the sentence will still be "correct". In many cases, substitutability arises from similarity (crunchy, crispy) but it does also arise from opposites. You may consider "king" and "...

12

I agree with Sean's answer. NLP and text mining are usually used for different goals. Also, there is indeed an overlap and both definitions are vogue. Other than the difference in goal, there is a difference in methods. Text mining techniques are usually shallow and do not consider the text structure. Usually, text mining will use bag-of-words, n-grams and ...

Only top voted, non community-wiki answers of a minimum length are eligible