Questions tagged [natural-language-process]

Natural language processing (NLP) is a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages. As such, NLP is related to the area of human–computer interaction. Many challenges in NLP involve natural language understanding, that is, enabling computers to derive meaning from human or natural language input, and others involve natural language generation. See NLP.

Filter by
Sorted by
Tagged with
0
votes
1answer
11 views

Is LSTM or pretrained BERTForMasked LM usable for predicting changed word in a sentence using a small dataset? (2000 samples)

I have a small (2000 samples) dataset of newspaper headlines and their humorous conterparts where only one word is changed to sound silly, for example: Original headline: Police <officer> ...
0
votes
0answers
10 views

Language Model with Attention not learning

Language model with attention layer is not learning after 20 epochs. Both training and validation loss increase together, while the accuracy flattens at around 7% The way input data is pipelined is by ...
0
votes
0answers
20 views

Training a Supervised Classifier using Continuous Data

I am trying to build a NLP classification model using methods such as XGBoost, SVM, logistic regression. The features I am trying to include are cosine similarity and LDA topic models, all of which ...
0
votes
1answer
14 views

Difference between zero-padding and character-padding in Recurrent Neural Networks

For RNN's to work efficiently we vectorize the problem which results in an input matrix of shape (m, max_seq_len) where m is the number of examples, e.g. ...
0
votes
0answers
6 views

Multilingual alternatives for med7

I'm looking for alternatives for med7 library for other common languages. Training a custom NER model for different languages seems like not the right option to ...
2
votes
3answers
48 views

Which machine learning problem is this?

I am not able to figure out what kind of machine learning is this: Training set: consists of sentences with object labels for object phrases Example: ...
1
vote
1answer
25 views

Preparing training data for NLP machine learning task

I have the natural language sentences as follows: This is a black chair. It is next to the table. Each phrase that represents an object is annotated with an object ...
2
votes
0answers
16 views

Medical NER for French language

I'm currently exploring the options to extract medical NER specifically for French language. I tried SpaCy's general French NER but it wasn't helpful to the cause (...
1
vote
0answers
36 views

Learning words embedding for bigrams and unigrams in a corpus

I am working on a topic modeling for tweets projects. I have generated my topics using both unigrams and bigrams. Topics are defined with a mixture of both bigrams and unigrams. Now I am planning to ...
0
votes
0answers
19 views

Is it possible to use unlabeled text articles for summarization when fine tuning BERT?

I know that unlabeled data could be used in pre-training but if I want to do a fine tuning of unlabeled articles for summarization, is it mandatory that the articles are labeled with existing ...
3
votes
1answer
24 views

how to programmatically introduce grammatical errors in sentences

I've a set of sentences in English language. I'm exploring ways to create a dataset of sentences with grammatical errors programmatically. The following options has been tried out randomly - identify ...
1
vote
1answer
16 views

How to handle like meaning sentences when working on text summarization

Suppose we have a text like Today is a very bad day. Very bad day is today. I wont come to play. What kind of technique should I use to summarize similar texts like ...
4
votes
2answers
158 views

Converting paragraphs into sentences

I'm looking for ways to extract sentences from paragraphs of text containing different types of punctuations and all. I used SpaCy's ...
0
votes
0answers
14 views

How to correctly lemmatize the text column in R?

I'm working on a project in Natural Language Processing. I have a data frame that has a text column. I have to lemmatize that text column. I'm using lemmatize_strings() function in R. However, there's ...
2
votes
0answers
39 views

Is it acceptable to append information to word embeddings?

Let's say I have my 300 dimensional word embedding trained with Word2Vec and it contains 10,000 word vectors. I have additional data on the 10,000 words in the form of a vector (10,000x1), containing ...
1
vote
0answers
10 views

auto updating text comparison model

I have a need to create a model that compares and groups distinct snippets of text based on keywords. I can extract similar keywords with NLP methods and simply comparing sentence text. I want these ...
3
votes
0answers
53 views

Word2Vec: Why do some dimensions of an embedding have an interpretation, and why does addition/subtraction of embedding vectors work?

I'm reading about Word2Vec from this source: http://jalammar.github.io/illustrated-word2vec/. Below is the heatmap of the embeddings for various words. In the source, it's claimed that we can get an ...
3
votes
2answers
81 views

Is NLP suitable for my legal contract parsing problem?

My company has a product that involves the extraction of a variety of fields from legal contract PDFs. The current approach is very time consuming and messy, and I am exploring if NLP is a suitable ...
1
vote
2answers
33 views

Bag-of-words and Spam classifiers

I implemented a spam classifier using Bernoulli Naive Bayes, Logistic Regression, and SVM. Algorithms are trained on the entire Enron spam emails dataset using the Bag-of-words (BoW) approach. ...
0
votes
0answers
23 views

How to add time as a feature into word embeddings?

I have a text corpus and I'm using TfidfVectorizer. Would it be possible to cluster the resultant matrix once I concatenate tf-idf vector and time feature matrix I built([year, month, day])? I'm also ...
0
votes
0answers
7 views

Projection layer function

I am trying to understand word-vectors and was reading this paper https://arxiv.org/pdf/1301.3781.pdf This paper proposes CBOW architecture which uses projection layer. What is projection layer? I ...
0
votes
0answers
15 views

What are the different ways to feature engineer webpage data for input into a webpage classification model?

Looking for resources on the different ways that one can manipulate webpage data to input as features into a neural net. I'm aware of a service called diffbot that claims to use a CV based method to &...
0
votes
1answer
28 views

Optimal input setup for character-level text classification RNN

I want to classify 500-character long text samples as to whether they look like natural language using a character-level RNN. I'm unsure as to the best way to feed the input to the RNN. Here are two ...
0
votes
0answers
8 views

Research on Product Search Machine Learning @ Amazon

I am looking for any sort of research paper/ reference where we can read more about machine learning principles using for the amazon's main website. Mainly interested in how they do the product search ...
1
vote
1answer
26 views

Working Behavior of BERT vs Transformers vs Self-Attention+LSTM vs Attention+LSTM on the scientific STEM data classification task?

So I just used BERT pre-trained with Focal Loss to classify Physics, Chemistry, Biology and Mathematics and got a good f-1 macro of 0.91. It is good given it only had to look for the tokens like ...
0
votes
0answers
13 views

Converting a string to a recommendation type string

I am trying to build a recommendation system and some of the labels are ...
1
vote
1answer
39 views

From where does BERT get the tokens it predicts?

When BERT is used for masked language modeling, it masks a token and then tries to predict it. What are the candidate tokens BERT can choose from? Does it just predict an integer (like a regression ...
0
votes
0answers
13 views

How one can match similar sounding features of two items

I am trying to build simple app which will compare two products from provideid URLs based on their features. We will be showing the products in tabular form with columns being each product and rows ...
2
votes
1answer
24 views

Bug in sentiment analysis and classification for unlabeled text [closed]

I'm working on the transcript of Trump and Biden's debate and want to analyze the sentences and classify negative, positive, or neutral comments, but I ran into one problem. I used both TextBlob and ...
1
vote
1answer
44 views

Why BERT tokenizers function differently?

While experimenting with transformers' TFBertForSequenceClassification and BertTokenizer, I noticed that BertTokenizer: ...
0
votes
0answers
20 views

Question about Relative-Position-Representation code

In https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/layers/common_attention.py In _relative_attention_inner method, which I think is one of the ...
2
votes
2answers
54 views

Evaluating Language Model on specific topic

I have finetuned a pretrained Language Model(GPT-2) on a custom dataset of mine. I would like a way of evaluating the ability of my model to generate sentences of a specific predefined topic, given in ...
0
votes
1answer
46 views

Ordering of standardization, pca, and/or tfidf for neural network

I have 60k rows of text data. I have tokenized it into 55k columns. I am using a neural network to classify the data but have some questions about how to order my preprocessing steps. I have too much ...
1
vote
0answers
21 views

How to process list type questions in Question Answering task [closed]

How to generate question-answer-context triplets for questions with multiple answer strings? How to measure performance for it? For a question with one single answer, we generate one question-answer-...
2
votes
1answer
47 views

What's the best way to detect bible verse mentions in a text?

I have a set of 10 verses from the Bible in English. I want to detect the occurrence of any of these verses in a text. What would be the best way to go about doing this? Note that verses of the Bible ...
1
vote
1answer
105 views

Question about BERT embeddings with high cosine similarity

Under what circumstances would BERT assign two occurrences of the same word similar embeddings? If those occurrences are contained within similar syntactic relations with their co-occurrents?
1
vote
0answers
14 views

Trying to use CBOW for tweet classification

I'm trying to use the Continuous Bag Of Words method for word embedding on a corpus of 7503 tweets. In particular, I'm trying to use CBOW on this Kaggle competition, which involves classifying tweets ...
-2
votes
1answer
26 views

I want to start studying the field of machine translation [closed]

I've studied Japanese language and literature and passed some linguistic courses and now as for my masters, I want to study natural language processing and especially machine translation. so I tried ...
0
votes
1answer
32 views

How to handle Tokenized text content which is given in number?

i have one data set of customer review, but the text data is given is tokenized text number. I am unable to proceed thinking about how to proceed? As I am encountering such data set the first time, so ...
0
votes
1answer
29 views

Is it possible to classify documents of corpus using labels?

I have a corpus of 23000 documents that need to be classified into 5 different categories. I do not have any labeled data available to me, just freeform text documents and labels(yes, one-word labels, ...
3
votes
2answers
101 views

How is the Gaussian noise given to this BLSTM based GAN?

In a conditional GAN, we give a random noise along with a label to the generator as input. In this paper, I don't understand why in one section of the paper, they say they are giving the random noise ...
2
votes
1answer
27 views

Word representation that gives more weight to terms frequent in corpus?

The tf-idf discounts the words that appear in a lot of documents in the corpus. I am constructing an anomaly detection text classification algorithm that is trained only on valid documents. Later I ...
1
vote
2answers
55 views

State-of-the-art Python packages that can evaluate language similarity

I am trying to evaluate the likelihood of generating a specific sentence out of a large set of sentences. To do this, I start from a simple approach: training a custom n-gram language model and ...
2
votes
4answers
109 views

NLP SBert (Bert) for answer comparison STS

I've been researching a good way to automate short answer evaluation. Essentially a teacher gives a test with some questions like: Question: why did columbus sail westward to find asia? Answer: so he ...
0
votes
1answer
29 views

Is adding the embedded words of a sentence to represent the sentence a good approach?

I have a dataset of sentences in a non english language like : word1 word2 word3 word62 word5 word1 word2 Now i want to turn each variable length sentence to a fixed size vector to give it to my ...
1
vote
1answer
64 views

Based on transformer, how to improve the text generation results?

If I do not pretrain the text generation model like BART, how to improve the result based on transformer like tensor2tensor? What are the improvement ideas for transformer in text generation task?
0
votes
0answers
27 views

Are there any deep models that are better than BERT-CRF in NER task?

Named entity recognition (NER) is task that mark tags of the input text sequence. BERT-CRF is a good NER model. I want to find a better NER model. Or I want to improve the BERT-CRF model. What can I ...
0
votes
1answer
36 views

best approach to embed random length sequences of words as a fixed size vector without having a maximum length? [closed]

I have a dataset of sentences in a non-English language like: word1 word2 word3 word62 word5 word1 word2 and the length of each sentence is not fixed. Now, I want to represent each sentence as a ...
0
votes
1answer
41 views

Using BERT for co-reference resolving, what's the loss function?

I'm working my way around using BERT for co-reference resolving. I'm following this highly-cited paper BERT for Coreference Resolution: Baselines and Analysis (https://arxiv.org/pdf/1908.09091.pdf). I ...

1
2 3 4 5 6