Questions tagged [nlp]

Natural language processing (NLP) is a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages. As such, NLP is related to the area of human–computer interaction. Many challenges in NLP involve natural language understanding, that is, enabling computers to derive meaning from human or natural language input, and others involve natural language generation.

Filter by
Sorted by
Tagged with
70 votes
4 answers
58k views

What is the positional encoding in the transformer model?

I'm trying to read and understand the paper Attention is all you need and in it, there is a picture: I don't know what positional encoding is. by listening to some youtube videos I've found out that ...
user avatar
  • 803
61 votes
6 answers
29k views

Latent Dirichlet Allocation vs Hierarchical Dirichlet Process

Latent Dirichlet Allocation (LDA) and Hierarchical Dirichlet Process (HDP) are both topic modeling processes. The major difference is LDA requires the specification of the number of topics, and HDP ...
user avatar
  • 2,372
37 votes
5 answers
11k views

What are some standard ways of computing the distance between documents?

When I say "document", I have in mind web pages like Wikipedia articles and news stories. I prefer answers giving either vanilla lexical distance metrics or state-of-the-art semantic distance metrics,...
user avatar
  • 791
36 votes
4 answers
29k views

What is purpose of the [CLS] token and why is its encoding output important?

I am reading this article on how to use BERT by Jay Alammar and I understand things up until: For sentence classification, we’re only only interested in BERT’s output for the [CLS] token, so we ...
user avatar
33 votes
6 answers
65k views

How do I load FastText pretrained model with Gensim?

I tried to load fastText pretrained model from here Fasttext model. I am using wiki.simple.en ...
user avatar
33 votes
6 answers
80k views

Sentence similarity prediction

I'm looking to solve the following problem: I have a set of sentences as my dataset, and I want to be able to type a new sentence, and find the sentence that the new one is the most similar to in the ...
user avatar
  • 1,248
31 votes
5 answers
36k views

How can I get a measure of the semantic similarity of words?

What is the best way to figure out the semantic similarity of words? Word2Vec is okay, but not ideal: ...
user avatar
30 votes
4 answers
29k views

What algorithms should I use to perform job classification based on resume data?

Note that I am doing everything in R. The problem goes as follow: Basically, I have a list of resumes (CVs). Some candidates will have work experience before and some don't. The goal here is to: ...
user avatar
30 votes
3 answers
32k views

General approach to extract key text from sentence (nlp)

Given a sentence like: Complimentary gym access for two for the length of stay ($12 value per person per day) What general approach can I take to identify the ...
user avatar
28 votes
4 answers
24k views

When to use cosine simlarity over Euclidean similarity

In NLP, people tend to use cosine similarity to measure document/text distances. I want to hear what do people think of the following two scenarios, which to pick, cosine similarity or Euclidean? ...
user avatar
  • 403
28 votes
7 answers
48k views

How to get sentence embedding using BERT?

How to get sentence embedding using BERT? ...
user avatar
  • 1,131
28 votes
8 answers
53k views

Best practical algorithm for sentence similarity

I have two sentences, S1 and S2, both which have a word count (usually) below 15. What are the most practically useful and successful (machine learning) algorithms, which are possibly easy to ...
user avatar
  • 433
28 votes
3 answers
19k views

What is a better input for Word2Vec?

This is more like a general NLP question. What is the appropriate input to train a word embedding namely Word2Vec? Should all sentences belonging to an article be a separate document in a corpus? Or ...
user avatar
  • 3,220
27 votes
3 answers
2k views

Why are NLP and Machine Learning communities interested in deep learning?

I hope you can help me, as I have some questions on this topic. I'm new in the field of deep learning, and while I did some tutorials, I can't relate or distinguish concepts from one another.
user avatar
27 votes
1 answer
33k views

Word2Vec vs. Sentence2Vec vs. Doc2Vec

I recently came across the terms Word2Vec, Sentence2Vec and Doc2Vec and kind of confused as I am new to vector semantics. Can someone please elaborate the differences in these methods in simple words. ...
user avatar
  • 499
26 votes
1 answer
8k views

NLP - why is "not" a stop word?

I am trying to remove stop words before performing topic modeling. I noticed that some negation words (not, nor, never, none etc..) are usually considered to be stop words. For example, NLTK, spacy ...
user avatar
  • 365
26 votes
2 answers
21k views

Predicting a word using Word2vec model

Given a sentence: "When I open the ?? door it starts heating automatically" I would like to get the list of possible words in ?? with a probability. The basic concept used in word2vec model ...
user avatar
  • 325
26 votes
4 answers
13k views

Word2Vec for Named Entity Recognition

I'm looking to use google's word2vec implementation to build a named entity recognition system. I've heard that recursive neural nets with back propagation through structure are well suited for named ...
user avatar
  • 2,009
25 votes
5 answers
17k views

Improve the speed of t-sne implementation in python for huge data

I would like to do dimensionality reduction on nearly 1 million vectors each with 200 dimensions(doc2vec). I am using TSNE ...
user avatar
  • 1,924
24 votes
5 answers
33k views

Natural Language to SQL query

I have been working on developing a system "Converting Natural Language to SQL Query". I have read the answers from the similar questions, but was not able to get the information that I was looking ...
user avatar
  • 1,391
22 votes
5 answers
4k views

How to annotate text documents with meta-data?

Having a lot of text documents (in natural language, unstructured), what are the possible ways of annotating them with some semantic meta-data? For example, consider a short document: ...
user avatar
21 votes
3 answers
4k views

How to grow a list of related words based on initial keywords?

I recently saw a cool feature that was once available in Google Sheets: you start by writing a few related keywords in consecutive cells, say: "blue", "green", "yellow", and it automatically generates ...
user avatar
21 votes
2 answers
14k views

NLP - Is Gazetteer a cheat?

In NLP, there is the concept of Gazetteer which can be quite useful for creating annotations. As far as I understand: A gazetteer consists of a set of lists ...
user avatar
  • 368
20 votes
4 answers
39k views

How to initialize a new word2vec model with pre-trained model weights?

I am using Gensim Library in python for using and training word2vector model. Recently, I was looking at initializing my model weights with some pre-trained word2vec model such as (GoogleNewDataset ...
user avatar
  • 451
20 votes
2 answers
4k views

What is the bleu score of professional human translators?

Machine translation models are usually evaluated using bleu score. I want to get some intuition for this score. What is the bleu score of professional human translator? I know it depends on the ...
user avatar
20 votes
3 answers
10k views

Dataset for Named Entity Recognition on Informal Text

I'm currently searching for labeled datasets to train a model to extract named entities from informal text (something similar to tweets). Because capitalization and grammar are often lacking in the ...
user avatar
  • 2,009
19 votes
6 answers
72k views

Similarity between two words

I'm looking for a Python library that helps me identify the similarity between two words or sentences. I will be doing Audio to Text conversion which will result in an English dictionary or non ...
user avatar
  • 709
19 votes
3 answers
41k views

What is the difference between CountVectorizer token counts and TfidfTransformer with use_idf set to False?

We can use CountVectorizer to count the number of times a word occurs in a corpus: ...
user avatar
18 votes
2 answers
23k views

Extract most informative parts of text from documents

Are there any articles or discussions about extracting part of text that holds the most of information about current document. For example, I have a large corpus of documents from the same domain. ...
user avatar
  • 631
18 votes
4 answers
23k views

What is the difference between a hashing vectorizer and a tfidf vectorizer

I'm converting a corpus of text documents into word vectors for each document. I've tried this using a TfidfVectorizer and a HashingVectorizer I understand that a ...
user avatar
  • 735
18 votes
4 answers
4k views

In a Transformer model, why does one sum positional encoding to the embedding rather than concatenate it?

While reviewing the Transformer architecture, I realized something I didn't expect, which is that : the positional encoding is summed to the word embeddings rather than concatenated to it. ...
user avatar
17 votes
3 answers
15k views

What is the difference between word-based and char-based text generation RNNs?

While reading about text generation with Recurrent Neural Networks I noticed that some examples were implemented to generate text word by word and others character by character without actually ...
user avatar
  • 1,987
16 votes
5 answers
7k views

Are there any good out-of-the-box language models for python?

I'm prototyping an application and I need a language model to compute perplexity on some generated sentences. Is there any trained language model in python I can readily use? Something simple like <...
user avatar
  • 393
15 votes
3 answers
26k views

Word2Vec how to choose the embedding size parameter

I'm running word2vec over collection of documents. I understand that the size of the model is the number of dimensions of the vector space that the word is embedded into. And that different dimensions ...
user avatar
  • 237
15 votes
1 answer
18k views

What is a 1D Convolutional Layer in Deep Learning?

I have a good general understanding of the role and mechanism of convolutional layers in Deep Learning for image processing in case of 2D or 3D implementations - they "simply" try to catch 2D patterns ...
user avatar
  • 7,787
15 votes
3 answers
1k views

How does attention mechanism learn?

I know how to build an attention in neural networks. But I don’t understand how attention layers learn the weights that pay attention to some specific embedding. I have this question because I’m ...
user avatar
14 votes
2 answers
30k views

Word2Vec embeddings with TF-IDF

When you train the word2vec model (using for instance, gensim) you supply a list of words/sentences. But there does not seem to be a way to specify weights for the words calculated for instance using ...
user avatar
  • 261
14 votes
4 answers
16k views

How word2vec can be used to identify unseen words and relate them to already trained data

I was working on word2vec gensim model and found it really interesting. I am intersted in finding how a unknown/unseen word when checked with the model will be able to get similar terms from the ...
user avatar
  • 341
13 votes
3 answers
29k views

What is the difference between NLP and text mining?

As discussed with Sean in this Meta post, I thought it would be nice to have a question which can help people who were confused like me, to know about the differences between text mining and NLP! So, ...
user avatar
  • 7,996
13 votes
4 answers
3k views

Extract information from sentence

I'm creating a simple chatbot. I want to obtain the information from the user response. An example scenario: Bot : Hi, what is your name? User: My name is Edwin. ...
user avatar
  • 323
13 votes
3 answers
3k views

Unsupervised feature learning for NER

I have implemented NER system with the use of CRF algorithm with my handcrafted features that gave quite good results. The thing is that I used lots of different features including POS tags and lemmas....
user avatar
  • 631
13 votes
1 answer
6k views

So what's the catch with LSTM?

I am expanding my knowledge of the Keras package and I have been tooling with some of the available models. I have an NLP binary classification problem that I'm trying to solve and have been applying ...
user avatar
13 votes
2 answers
4k views

What features are generally used from Parse trees in classification process in NLP?

I am exploring different types of parse tree structures. The two widely known parse tree structures are a) Constituency based parse tree and b) Dependency based parse tree structures. I am able to ...
user avatar
  • 231
13 votes
4 answers
10k views

Alternatives to TF-IDF and Cosine Similarity when comparing documents of differing formats

I've been working on a small, personal project which takes a user's job skills and suggests the most ideal career for them based on those skills. I use a database of job listings to achieve this. At ...
user avatar
12 votes
3 answers
3k views

Efficient database model for storing data indexed by n-grams

I'm working on an application which requires creating a very large database of n-grams that exist in a large text corpus. I need three efficient operation types: Lookup and insertion indexed by the n-...
user avatar
  • 288
12 votes
1 answer
9k views

Why do we need to add START <s> + END </s> symbols when using Recurrent Neural Nets for Sequence-to-Sequence Models?

In the Sequence-to-Sequence models, we often see that the START (e.g. <s>) and END (e.g. </s>) symbols are added to ...
user avatar
  • 2,372
12 votes
3 answers
6k views

Help regarding NER in NLTK

I have been working in NLTK for a while using Python. The problem I am facing is that their is no help available on training NER in NLTK with my custom data. They have used MaxEnt and trained it on ...
user avatar
  • 195
12 votes
4 answers
9k views

How to process natural language queries?

I'm curious about natural language querying. Stanford has what looks to be a strong set of software for processing natural language. I've also seen the Apache OpenNLP library, and the General ...
user avatar
12 votes
2 answers
6k views

Variable input/output length for Transformer

I was reading the paper "Attention is all you need" (https://arxiv.org/pdf/1706.03762.pdf ) and came across this site http://jalammar.github.io/illustrated-transformer/ which provided a great ...
user avatar
  • 241
12 votes
1 answer
406 views

ngram and RNN prediction rate wrt word index

I tried to plot the rate of correct predictions (for the top 1 shortlist) with relation to the word's position in sentence : I was expecting to see a plateau sooner on the ngram setup since it ...
user avatar
  • 157

1
2 3 4 5
47