Questions tagged [nlp]

Natural language processing (NLP) is a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages. As such, NLP is related to the area of human–computer interaction. Many challenges in NLP involve natural language understanding, that is, enabling computers to derive meaning from human or natural language input, and others involve natural language generation.

Filter by
Sorted by
Tagged with
0
votes
3answers
28 views

Distinguish randomly generated texts from reasonable for human texts

I have strings short texts of 2 types: '23jd2032n0d2mn', 'fn830n30rn83', 'fhui29n4ok', 'qn4foml', ... and ...
0
votes
0answers
8 views

How to find 'pre-requisite' relationships between sentences

I have two arrays. Each of them is consists of some sentences indicating some action. For example: ...
0
votes
0answers
15 views

What is the typical accuracy of masked language models during BERT pretraining?

I was reading the BERT paper but I didn't find any tables concerning the performance of the masked language models during pretraining. Does anyone know the accuracy of BERT's masked language model?
1
vote
0answers
18 views

Learning word embeddings by first learning character embeddings

I was going through various papers for NLU applications(Natural Language Understanding). There I have observed a common pattern that for a word embeddings, following 3 combinations are used (may be ...
1
vote
0answers
14 views

How to process list type questions in Question Answering task

How to generate question-answer-context triplets for questions with multiple answer strings? How to measure performance for it? For a question with one single answer, we generate one question-answer-...
0
votes
0answers
19 views

What would be the target input for Transformer Decoder during test phase?

The Transformer Decoder takes in two inputs, the encoder's output, and the target sequence. How the target is fed into the decoder has been provided in this answer I am having confusion about what ...
0
votes
1answer
20 views

How to approach for predicting semantic similarity between two phrases

I need pointers on the latest research, tools, and techniques for predicting semantic similarity between two phrases. Problem Statement: Given two propositions A ...
2
votes
0answers
21 views

What's the best way to detect bible verse mentions in a text?

I have a set of 10 verses from the Bible in English. I want to detect the occurrence of any of these verses in a text. What would be the best way to go about doing this? Note that verses of the Bible ...
0
votes
2answers
24 views

how can i detect medicine name and info(use and contents) by using medicine wrappers

I got one project idea creating a Cross-platform react-native app the project title is creating an app that can detect medicine name and other info from the medicine wrapper I'm thinking of using ...
0
votes
0answers
8 views

How to improve accuracy for model dan val?

...
1
vote
0answers
21 views

Does BERT has any advantage over GPT3?

I have read a couple of documents that explain in detail about the greater edge that GPT-3(Generative Pre-trained Transformer-3) has over BERT(Bidirectional Encoder Representation from Transformers). ...
0
votes
1answer
15 views

Marginal contribution of a text document

I'm trying to build a Shapley value (marginal contribution) of a text document in terms of information content, given that there are several documents on a given topic. For example, we have 3 reports ...
0
votes
0answers
29 views

What is the best way to train an intent classifier with imbalanced data?

I am trying to identify a particular intent in a corpus of text messages. The data is very imbalanced (maybe 1000 true labels in a 200 000 text messages). Out of those intents, I also need to make ...
1
vote
0answers
24 views

Question about BERT embeddings with high cosine similarity

Under what circumstances would BERT assign two occurrences of the same word similar embeddings? If those occurrences are contained within similar syntactic relations with their co-occurrents?
-2
votes
0answers
15 views
0
votes
1answer
11 views

Semantic similarity between two or more sentences

I need to determine how similar sentences (in meaning) are to one another. In order to do it, I have been considering an algorithm (cosine similarity) to determine the similarity between sentences. I ...
0
votes
1answer
16 views

How to determine semantic differences in NLP

I would need to determine the difference in meaning between the following two sentences: I am at home I am not at home I am at the office the first two sentences ...
0
votes
2answers
41 views

How to extract contents by topic from a document?

I am trying to extract information from resumes. I tried the pdfminer for the text extraction. But I need to extract the contents from a resume with respect to its title. For example: I will be giving ...
1
vote
0answers
9 views

Data Annotation: “labeling” target vs features

I understand how one would use a data annotation tool to label targets for a given sentence, for example though, I'm not clear on how placing labels on features can be used to improve model ...
1
vote
0answers
29 views

GPT-3 API Documentation?

Has a documentation of the GPT-3 API been made public? I would be interested in keeping myself up to speed on the API's capability. Thanks!
2
votes
2answers
37 views

Sentiment analysis of the target in articles

The goal is as follows: I have a big article and I want to define the sentiment of the particular word. For example, the article describes pros and cons of bikes and cars and I want to find the ...
1
vote
1answer
16 views

Construct word2vec (CBOW) training data from beginning of sentence

When constructing training data for CBOW, Mikolov et al. suggest using the word from the center of a context window. What is the "best" approach to capturing words at the beginning/end of a ...
1
vote
1answer
26 views

Does finetuning BERT involving updating all of the parameters or just the final classification layer?

Currently learning and reading about transformer models, I get that during the pretraining stage the BERT model is trained on a large corpus via MLM and NSP. But during finetuning, for example trying ...
0
votes
0answers
7 views

Completely different results using polarity (qdap) and sentimentr [closed]

I have run some survey responses through polarity analysis in qdap. I then ran the same through sentimentr and found completely different results. Not even close. I don't know enough about NLP to ...
0
votes
0answers
19 views

How can I get a value of context vector in GPT?

I'm a newbie in NLP and I'm now stuck in GPT. The question I'm struggling with is related to a term 'context vector' It says in the following (sorry that the material provided is written in korean) ...
0
votes
0answers
22 views

Non-ML approach for defining components of the main aspect

Recently I started studying NLP and trying to make a pet project the goal of which is sentiment identification for particular word. Let's imagine I have a text like "Ford SUVs have repeatedly won ...
0
votes
2answers
43 views

Getting sentence embeddings with sentence_transformers

I have a text column in my data frame which contains paragraph(s) having multiple and variable sentences in each instance/example/row of the dataframe. Then, I created the sentence tokens of that ...
0
votes
1answer
15 views

How to expand abbrevations in text during preprocessing?

Im doing preprocessing on english text data. I have some domain specific abbreviations, for which i'm maintaining internal dictionary with key-value pairs. The problem i'm facing is the text has ...
0
votes
1answer
20 views

Is it good practice to remove the numeric values from the text data during preprocessing?

Im doing preprocessing on a text dataset. I have certain numerics in it like: date(1st July) year(2019) tentative values (3-5 years/ 10+ advantages). unique values (room no 31/ user rank 45) ...
1
vote
3answers
33 views

How to process the hyphenated english words for any nlp problem?

Im doing preprocessing on english text dataset. I encounter hyphenated words like 'well-known'. Will it be useful if I remove the hyphen as special character and treat it as a single word 'wellknown' ...
0
votes
0answers
16 views

How is Large BERT is less accurate than basic BERT?

I'm using BERT for text classification in this NLP competition. When I use Basic BERT with 12 layers, 3 epochs, and 32 batch sizes, I get a training accuracy of about 0.84 and a val_accuracy of about ...
1
vote
1answer
39 views

How to properly compare these two confusion matrix?

I have used Vader, a sentiment analysis tool for social media, on a database of movie reviews. These two confusion matrices differ in the vader.py algorithm, as the first one is from nltk: The second ...
0
votes
0answers
16 views

Sampling methods for Text datasets (NLP)

I am working on two text datasets, one is having 68k text samples and other is having 100k text samples. I have encoded the text datasets into bert embedding. ...
2
votes
0answers
46 views

Loss first decreases and then increases

I am using pre-trained xlnet-base-cased model and training it further on real vs fake news detection dataset. I noticed a trend in accuracy for first epoch. ...
0
votes
1answer
28 views

How to handle Tokenized text content which is given in number?

i have one data set of customer review, but the text data is given is tokenized text number. I am unable to proceed thinking about how to proceed? As I am encountering such data set the first time, so ...
0
votes
0answers
13 views

How to practically deal with unbalanced data and realistic class distributions in a Naive Bayes classifier?

I’ve created a multi class multinomial Naive Bayes classifier for text classification. However my data is unbalanced which I’m sure is affecting the accuracy. For example class A has 500 datapoints ...
0
votes
0answers
9 views

How to tag Personal Name, Place and Organisation using Natural Language Processing or Fuzzy string matching

I have a data-set that has 2 columns in it: ...
0
votes
0answers
8 views

Building a Keras text embedding model with cosine proximity

I am trying to build a word embedding keras model wherein I give as input a text that is converted to its corresponding input ids and masks (like input to an Albert model) and it gives me back a 768 ...
0
votes
0answers
12 views

What is the difference between “Document Layout Analysis” / “Document Understanding” / “Document Structure Analysis”

All three terms sound super similar: [...] document layout analysis is the process of identifying and categorizing the regions of interest in the scanned image of a text document. A reading system ...
0
votes
0answers
14 views

Distance to the Center of Vocabulary of Word Embeddings

Suppose I: Generate a set of word embeddings (using word2vec or similar) based on a large but specific corpus Compute the centroid of all the words in the set Find the word(s) with the smallest (say ...
0
votes
0answers
13 views

How to Approach Creating an Accurate Multiclass Multinomial Naive Bayes with Unbalanced Data

I have used sklearn to create a basic multiclass naive bayes text classifier. I have 3 classes and around 800 rows of data. Class A has 564 rows, Class B has 159, and Class C has 82. As you can see ...
1
vote
0answers
20 views

Is summing a cosine similarity matrix a good way to determine overall similarity?

I'm trying to similar research abstracts, so I'm using word embeddings to convert words into 1x768 vectors, so overall turning abstracts into embeddings with shape (#ofwords, 768). Cosine similarity ...
0
votes
2answers
28 views

Is it bad to have a lot of one class of Data [K-NN classifier]?

I am trying to train a sklearn K-NN classifier on a labeled text dataset (in Irish). There are 5 classes, 0-4, but there is a lot of variation between how many there are in each class. What I have ...
2
votes
1answer
21 views

Transformer masking during training or inference?

I'm working through Attention is All you Need, and I have a question about masking in the decoder. It's stated that masking is used to ensure the model doesn't attend to any tokens in the future (not ...
1
vote
1answer
23 views

TF-IDF for Topic Modeling

Can TF-IDF be used a sole method for Topic Modeling ? (I know there are better methods like LDA , LSA etc) I just want to understand if TF-IDF alone can help us in Topic modeling . If yes , can ...
0
votes
0answers
13 views

What GPU size do I need to fine tune BERT base cased?

I want to fine tune BERT Multilingual but I'm not aware about the GPU requirements to train BERT Multilingual. I have GTX 1050ti 4GB on my local machine. I want to know what size of GPU is needed and ...
0
votes
1answer
29 views

Searching for a dataset that targets difficult words

I am trying to find a dataset in which dataset targets words that are difficult. I understand there would be different levels of difficulty for each individual , but if we considered an average ...
1
vote
1answer
28 views

Can we use sentence transformers to embed sentences without labels?

I was trying to use this project : https://github.com/UKPLab/sentence-transformers for embedding non english sentences, the language is not a human speaking language, its machine language (x86) but ...
0
votes
0answers
10 views

Manually tune tf-idf features in document classification

I am working on a multi-label document classification task with a very small data set (180 labeled documents) and a fairly large number of labels (20). I found that - ignoring label correlations and ...
0
votes
1answer
27 views

How can I picture an unfolded RNN as a normal Feed Forward Network?

I am currently working on a Transformer architecture. Trying to picture an RNN (or Encoder) as a normal Feed Forward network really confused me after looking at the following image in an article: (...

1
2 3 4 5
30