Questions tagged [nlp]

Natural language processing (NLP) is a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages. As such, NLP is related to the area of human–computer interaction. Many challenges in NLP involve natural language understanding, that is, enabling computers to derive meaning from human or natural language input, and others involve natural language generation.

Filter by
Sorted by
Tagged with
7 votes
4 answers
8k views

How to give name to topics created using LDA?

I have categorized 800,000 documents into 500 categories using the Mahout topic modelling. Instead of representing the topic using the top 5/10 words for each topics, I want to infer a generic name ...
user avatar
  • 71
3 votes
1 answer
5k views

How pre-trained BERT model generates word embeddings for out of vocabulary words?

Currently, I am reading BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. I want to understand how pre-trained BERT generates word embeddings for out of vocabulary ...
user avatar
6 votes
2 answers
7k views

What are useful evaluation metrics used in machine learning

I am using CNN in order to predict codes after analyzing text. As an example, I will write "I am crazy" .. the model will predict some code " X321". All this based on CNN. I want to evaluate my ...
user avatar
  • 215
26 votes
2 answers
21k views

Predicting a word using Word2vec model

Given a sentence: "When I open the ?? door it starts heating automatically" I would like to get the list of possible words in ?? with a probability. The basic concept used in word2vec model ...
user avatar
  • 325
18 votes
4 answers
4k views

In a Transformer model, why does one sum positional encoding to the embedding rather than concatenate it?

While reviewing the Transformer architecture, I realized something I didn't expect, which is that : the positional encoding is summed to the word embeddings rather than concatenated to it. ...
user avatar
15 votes
1 answer
18k views

What is a 1D Convolutional Layer in Deep Learning?

I have a good general understanding of the role and mechanism of convolutional layers in Deep Learning for image processing in case of 2D or 3D implementations - they "simply" try to catch 2D patterns ...
user avatar
  • 7,787
70 votes
4 answers
58k views

What is the positional encoding in the transformer model?

I'm trying to read and understand the paper Attention is all you need and in it, there is a picture: I don't know what positional encoding is. by listening to some youtube videos I've found out that ...
user avatar
  • 803
27 votes
3 answers
2k views

Why are NLP and Machine Learning communities interested in deep learning?

I hope you can help me, as I have some questions on this topic. I'm new in the field of deep learning, and while I did some tutorials, I can't relate or distinguish concepts from one another.
user avatar
33 votes
6 answers
65k views

How do I load FastText pretrained model with Gensim?

I tried to load fastText pretrained model from here Fasttext model. I am using wiki.simple.en ...
user avatar
21 votes
2 answers
14k views

NLP - Is Gazetteer a cheat?

In NLP, there is the concept of Gazetteer which can be quite useful for creating annotations. As far as I understand: A gazetteer consists of a set of lists ...
user avatar
  • 368
18 votes
2 answers
23k views

Extract most informative parts of text from documents

Are there any articles or discussions about extracting part of text that holds the most of information about current document. For example, I have a large corpus of documents from the same domain. ...
user avatar
  • 631
13 votes
4 answers
10k views

Alternatives to TF-IDF and Cosine Similarity when comparing documents of differing formats

I've been working on a small, personal project which takes a user's job skills and suggests the most ideal career for them based on those skills. I use a database of job listings to achieve this. At ...
user avatar
12 votes
3 answers
6k views

Help regarding NER in NLTK

I have been working in NLTK for a while using Python. The problem I am facing is that their is no help available on training NER in NLTK with my custom data. They have used MaxEnt and trained it on ...
user avatar
  • 195
8 votes
3 answers
5k views

Twitter Sentiment Analysis: Detecting neutral tweets despite training on only Positive and Negative Classes

I am a newbie when it comes to machine learning. I am trying to get hands on experience by analyzing different supervised learning algorithms using scikit-learn library of python. I am using the ...
user avatar
4 votes
3 answers
2k views

Categorizing Customer Emails

I am working on a project for a company which needs to categorize customer e-mails regarding loans and insurance. The e-mails are labeled uniquely from set of 13 category labels. The number of records ...
user avatar
1 vote
2 answers
227 views

How to obtain vector representation of phrases using the embedding layer and do PCA with it

I am trying to understand from both a conceptual and a Python code point of view, how to represent phrases that are present in a corpus (that is used to train a neural network to classify phrases) as ...
user avatar
4 votes
1 answer
1k views

Creating training data

My task is to classify free text originated from customer complaints about our product. I have created a Taxonomy and have around 10 different categories. I've realized that these categories include ...
user avatar
  • 709
2 votes
1 answer
41 views

Tweet Classification into topics- What to do with data

Good evening, First of all, I want to apologize if the title is misleading. I have a dataset made of around 60000 tweets, their date and time as well as the username. I need to classify them into ...
user avatar
0 votes
0 answers
138 views

Image segmentation network to extract questions from an image of a test paper?

This is the sample document -> I want to extract questions along with the options. There are other question papers as which have questions with diagrams in them. I want to be able to extract them ...
user avatar
0 votes
1 answer
279 views

How do the linear layers in the attention mechanism work?

I think I now the answer to my question but I dont really get confirmation. When taking a look at the multi-head-attention block as presented in "Attention Is All You Need" we can see that ...
user avatar
  • 123
10 votes
2 answers
6k views

Preprocessing for Text Classification in Transformer Models (BERT variants)

This might be silly to ask, but I am wondering if one should carry out the conventional text preprocessing steps for training one of the transformer models? I remember for training a Word2Vec or Glove,...
user avatar
  • 3,926
3 votes
1 answer
292 views

Comparing one small dataset with a big dataset for similar records

I create a varying small dataset (dataset: X) with 500 records in each query. Everytime I need to compare the dataset with a bigger one (dataset: A) (15 milion records) to find similar (or semi-...
user avatar
  • 143
30 votes
3 answers
32k views

General approach to extract key text from sentence (nlp)

Given a sentence like: Complimentary gym access for two for the length of stay ($12 value per person per day) What general approach can I take to identify the ...
user avatar
30 votes
4 answers
29k views

What algorithms should I use to perform job classification based on resume data?

Note that I am doing everything in R. The problem goes as follow: Basically, I have a list of resumes (CVs). Some candidates will have work experience before and some don't. The goal here is to: ...
user avatar
61 votes
6 answers
29k views

Latent Dirichlet Allocation vs Hierarchical Dirichlet Process

Latent Dirichlet Allocation (LDA) and Hierarchical Dirichlet Process (HDP) are both topic modeling processes. The major difference is LDA requires the specification of the number of topics, and HDP ...
user avatar
  • 2,372
33 votes
6 answers
80k views

Sentence similarity prediction

I'm looking to solve the following problem: I have a set of sentences as my dataset, and I want to be able to type a new sentence, and find the sentence that the new one is the most similar to in the ...
user avatar
  • 1,248
24 votes
5 answers
33k views

Natural Language to SQL query

I have been working on developing a system "Converting Natural Language to SQL Query". I have read the answers from the similar questions, but was not able to get the information that I was looking ...
user avatar
  • 1,391
27 votes
1 answer
33k views

Word2Vec vs. Sentence2Vec vs. Doc2Vec

I recently came across the terms Word2Vec, Sentence2Vec and Doc2Vec and kind of confused as I am new to vector semantics. Can someone please elaborate the differences in these methods in simple words. ...
user avatar
  • 499
31 votes
5 answers
36k views

How can I get a measure of the semantic similarity of words?

What is the best way to figure out the semantic similarity of words? Word2Vec is okay, but not ideal: ...
user avatar
28 votes
4 answers
24k views

When to use cosine simlarity over Euclidean similarity

In NLP, people tend to use cosine similarity to measure document/text distances. I want to hear what do people think of the following two scenarios, which to pick, cosine similarity or Euclidean? ...
user avatar
  • 403
14 votes
2 answers
30k views

Word2Vec embeddings with TF-IDF

When you train the word2vec model (using for instance, gensim) you supply a list of words/sentences. But there does not seem to be a way to specify weights for the words calculated for instance using ...
user avatar
  • 261
25 votes
5 answers
17k views

Improve the speed of t-sne implementation in python for huge data

I would like to do dimensionality reduction on nearly 1 million vectors each with 200 dimensions(doc2vec). I am using TSNE ...
user avatar
  • 1,924
15 votes
3 answers
26k views

Word2Vec how to choose the embedding size parameter

I'm running word2vec over collection of documents. I understand that the size of the model is the number of dimensions of the vector space that the word is embedded into. And that different dimensions ...
user avatar
  • 237
8 votes
2 answers
576 views

NLP : variations of a text without modifying it's meaning

I am currently working on the automation of recurring reports (weekly 30-50 pages reports for around 100 districts). Those reports have a mostly fixed form : maps, graphs, data tables and small zone ...
user avatar
  • 2,107
2 votes
1 answer
7k views

What is a good explanation of Non Negative Matrix Factorization?

I am trying to find a resource to understand non-negative matrix factorization. Apart from Wikipedia, I couldn't find anything useful.
user avatar
36 votes
4 answers
29k views

What is purpose of the [CLS] token and why is its encoding output important?

I am reading this article on how to use BERT by Jay Alammar and I understand things up until: For sentence classification, we’re only only interested in BERT’s output for the [CLS] token, so we ...
user avatar
21 votes
3 answers
4k views

How to grow a list of related words based on initial keywords?

I recently saw a cool feature that was once available in Google Sheets: you start by writing a few related keywords in consecutive cells, say: "blue", "green", "yellow", and it automatically generates ...
user avatar
20 votes
3 answers
10k views

Dataset for Named Entity Recognition on Informal Text

I'm currently searching for labeled datasets to train a model to extract named entities from informal text (something similar to tweets). Because capitalization and grammar are often lacking in the ...
user avatar
  • 2,009
11 votes
2 answers
13k views

How do "intent recognisers" work?

Amazon's Alexa, Nuance's Mix and Facebook's Wit.ai all use a similar system to specify how to convert a text command into an intent - i.e. something a computer would understand. I'm not sure what the "...
user avatar
  • 231
10 votes
1 answer
8k views

what is the first input to the decoder in a transformer model?

The image is from url: Jay Alammar on transformers K_encdec and V_encdec are calculated in a matrix multiplication with the encoder outputs and sent to the encoder-decoder attention layer of each ...
user avatar
8 votes
3 answers
7k views

Text classification with thousands of output classes in Keras

Task: I have a dataset with job titles and descriptions. The task is to predict tags for job by job title and description. There are several tags for each job posting. Therefore, the number of ...
user avatar
  • 195
22 votes
5 answers
4k views

How to annotate text documents with meta-data?

Having a lot of text documents (in natural language, unstructured), what are the possible ways of annotating them with some semantic meta-data? For example, consider a short document: ...
user avatar
12 votes
2 answers
6k views

Variable input/output length for Transformer

I was reading the paper "Attention is all you need" (https://arxiv.org/pdf/1706.03762.pdf ) and came across this site http://jalammar.github.io/illustrated-transformer/ which provided a great ...
user avatar
  • 241
10 votes
1 answer
1k views

How to determine the complexity of an English sentence?

I am working on an app to help people learn English as a second language. I have validated that sentences help in learning a language by providing extra context. I did that by conducting a small ...
user avatar
9 votes
4 answers
3k views

How to deal with spelling errors NLP

I have some data where the main column is the description of one product. The main task is to extract the name of some product from this column, where it sometimes is spelled wrong and amended in ...
user avatar
  • 221
6 votes
1 answer
2k views

Question and Answer Chatbot for Customer Support

I want to build a chatbot that serves as a first line customer support on a retail website. I have a large log of chat sessions between customers and support professionals that I can use. I am ...
user avatar
5 votes
2 answers
8k views

Text similarity using RNN

Data set contains records of short text, typically a sentence. The goal is to find duplicated records and similar records. Currently, I have tried R package 'text2vec', the glove word vectors and the ...
user avatar
16 votes
5 answers
7k views

Are there any good out-of-the-box language models for python?

I'm prototyping an application and I need a language model to compute perplexity on some generated sentences. Is there any trained language model in python I can readily use? Something simple like <...
user avatar
  • 393
11 votes
1 answer
3k views

applying word2vec on small text files

I'm totally new to word2vec so pls bear it with me. I have a set of text files each containing a set of tweets, between 1000-3000. I have chosen a common keyword ("kw1") and wants to find semantically ...
user avatar
  • 211
10 votes
5 answers
18k views

How to create a good list of stopwords

I am looking for some hints on how to curate a list of stopwords. Does someone know / can someone recommend a good method to extract stopword lists from the dataset itself for preprocessing and ...
user avatar
  • 333