# Tag Info

Accepted

### What are some standard ways of computing the distance between documents?

There's a number of different ways of going about this depending on exactly how much semantic information you want to retain and how easy your documents are to tokenize (html documents would probably ...
• 4,179
Accepted

### What is difference between text classification and topic models?

Text Classification I give you a bunch of documents, each of which has a label attached. I ask you to learn why you think the contents of the documents have been given these labels based on their ...
• 6,415

### General approach to extract key text from sentence (nlp)

Shallow Natural Language Processing technique can be used to extract concepts from sentence. ------------------------------------------- Shallow NLP technique steps: Convert the sentence to lowercase ...
Accepted

### Extract most informative parts of text from documents

What you're describing is often achieved using a simple combination of TF-IDF and extractive summarization. In a nutshell, TF-IDF tells you the relative importance of each word in each document, in ...
• 1,501

### How to annotate text documents with meta-data?

Personally I would advocate using something that is both not-specific to the NLP field, and something that is sufficiently general that it can still be used as a tool even when you've started moving ...
• 4,179
Accepted

### Keyword/phrase extraction from Text using Deep Learning libraries

The Google Research Blog should be helpful in the context of TensorFlow. In the above article, there is a reference to the Annotated English Gigaword dataset which is routinely used for text ...

### Keyword/phrase extraction from Text using Deep Learning libraries

This is an open area of research and it certainly depends on the way you frame the problem. If you're talking about multi-document summarization then the problem is slightly different than if you were ...
Accepted

### How to grow a list of related words based on initial keywords?

The word2vec algorithm may be a good way to retrieve more elements for a list of similar words. It is an unsupervised "deep learning" algorithm that has previously been demonstrated with Wikipedia-...
• 476
Accepted

### Suggest text classifier training datasets

Some standard datasets for text classification are the 20-News group, Reuters (with 8 and 52 classes) and WebKb. You can find all of them here.
• 1,516

### What algorithms should I use to perform job classification based on resume data?

Check out this link. Here, they will take you through loading unstructured text to creating a wordcloud. You can adapt this strategy and instead of creating a wordcloud, you can create a frequency ...
• 485

### Unstructured text classification

Let's work it out from the ground up. Classification (also known as categorization) is an example of supervised learning. In supervised learning you have: model - something that approximates ...
• 2,771
Accepted

### What are the main types of NLP annotators?

Here are the basic Natural Language Processing capabilities (or annotators) that are usually necessary to extract language units from textual data for sake of search and other applications: Sentence ...
• 301

### What is difference between text classification and topic models?

But I don't know what is difference between text classification and topic models in documents Text Classification is a form of supervised learning, hence the set ...
• 1,501
Accepted

### Algorithms for text clustering

Check the Stanford NLP Group's open source software, in particular, Stanford Classifier. The software is written in Java, which will likely delight you, but also ...
• 6,508
Accepted

### How to determine if character sequence is English word or noise

During NLP and text analytics, several varieties of features can be extracted from a document of words to use for predictive modeling. These include the following. ngrams Take a random sample of ...
• 1,206
Accepted

### What is the difference between NLP and text mining?

I agree with Sean's answer. NLP and text mining are usually used for different goals. Also, there is indeed an overlap and both definitions are vogue. Other than the difference in goal, there is a ...
• 2,583
Accepted

### Doc2Vec - How to label the paragraphs (gensim)

Both are possible. You can give every document a unique ID (such as a sequential serial number) as a doctag, or a shared string doctag representing something else about it, or both at the same time. ...
• 236
Accepted

### Resume Parsing - extracting skills from resume using Machine Learning

I'm not sure Topic Modelling will help you here, as it tries to extract abstract topics from text. I'm afraid resumes might be too 'dry' for it to work nicely. Here are a few sources I found that ...
• 136

### What algorithms should I use to perform job classification based on resume data?

Just extract keywords and train a classifier on them. That's all, really. Most of the text in CVs is not actually related to skills. E.g. consider sentence "I'm experienced and highly efficient in ...
• 2,771

### Doc2Vec - How to label the paragraphs (gensim)

doc2vec model gets its algorithm from word2vec. In word2vec there is no need to label the ...
• 1,924

### What is Hellinger Distance and when to use it?

Hellinger distance is a metric to measure the difference between two probability distributions. It is the probabilistic analog of Euclidean distance. Given two probability distributions, $P$ and $Q$, ...
• 16.5k

### Clustering with cosine similarity

First, every clustering algorithm is using some sort of distance metric. Which is actually important, because every metric has its own properties and is suitable for different kind of problems. You ...
• 1,641

### How to do postal addresses fuzzy matching?

As you are using R you might want to look into the stringdist package and the Jaro-Winkler distance metric that can be used in the calculations. This was developed at the U.S. Census Bureau for ...
• 720
Accepted

### What is the difference between a hashing vectorizer and a tfidf vectorizer

The main difference is that HashingVectorizer applies a hashing function to term frequency counts in each document, where ...
• 1,558
Accepted

### R error using package tm (text-mining)

You have to tell Corpus what kind of source you are using. Try: Corpus(VectorSource(d1\$Yes))
• 485

### Preference Matching Algorithm

My first suggestion would be to somehow map the non-quantifiable attributes to quantities with the help of suitable mapping functions. Otherwise, simply leave them out. Secondly, I don't think that ...
• 1,516

### applying word2vec on small text files

Word2Vec isn't a good choice for a dataset of such size. From researches I have seen, it will unleash its power if you feed at least couple of million of words, 3k tweets wouldn't be enough for a ...
• 779

### What is the difference between NLP and text mining?

I have had this doubt since a long time. So, this post here helped me figure the differences between the two. So, this is the difference between text mining and NLP: Text Mining deals with the ...
• 8,016