Questions tagged [nlp]

Natural language processing (NLP) is a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages. As such, NLP is related to the area of human–computer interaction. Many challenges in NLP involve natural language understanding, that is, enabling computers to derive meaning from human or natural language input, and others involve natural language generation.

Filter by
Sorted by
Tagged with
0 votes
1 answer
29 views

Training a model with a series of text responses as input

I want to train a binary classifier on text -- so something like sentiment analysis, but my input vectors are going to be a series of responses from some user separated by some separator character. I ...
user avatar
  • 133
0 votes
0 answers
9 views

Generate Legal Advice (Text Generation)

Somebody came to me with this use case/question and as I am not an expert yet on NLP, NLTK, spacy, GPT-x Then I dont know if this is possible and I would like to get some feedback. We have hundreds of ...
user avatar
2 votes
0 answers
29 views

Cluster tabular data with text in some columns

Let's say I have a following features in the my dataframe: user_id user_age is_student is_graduate salary resume integer integer binary binary integer text (up to 1000 symbols) And also a few more ...
user avatar
  • 21
1 vote
1 answer
62 views

What are the exact differences between Word Embedding and Word Vectorization?

I am learning NLP. I have tried to figure out the exact difference between Word Embedding and Word Vectorization. However, seems like some articles use these words interchangeably. But I think there ...
user avatar
  • 13
2 votes
1 answer
45 views

A multi label text classification problem

I'm looking to solve a multi label text classification problem but I don't really know how to formulate it correctly so I can look it up.. Here is my problem : Say I have the document ...
user avatar
  • 35
0 votes
0 answers
11 views

Tag texts using predefined keywords based on the importance

I want to tag a list of texts using predefined keywords ex: keyword1, keyword2, keyword3. I ...
user avatar
  • 101
1 vote
1 answer
37 views

Information Extraction from News Headlines

I'm relatively new in the field of Information Extraction and was wondering if there are any methods to summarize multiple headlines on the same topic, like some kind of "average" of ...
user avatar
  • 11
0 votes
0 answers
11 views

Using different layers in model output to achieve cosine distance in embedding space

I'm looking at the article of Sentence-BERT, I'm trying to do some embeddings with the same siamese architecture. Later, I will want to compare embeddings from my model with FastText model using ...
user avatar
  • 1
1 vote
0 answers
14 views

Classifier as compression

Someone wrote a Wordle clone in bash, using the full system dictionary. Unfortunately there is no smaller list of "common words" available locally to make use of. I was wondering if a ...
user avatar
3 votes
2 answers
40 views

How to do sentence segmentation without loosing sentence's subject?

I have some text with different lengths, I want to split it into separate clauses but I also want to preserve the subject For example; ...
user avatar
1 vote
1 answer
25 views

How to deal with "Ergänzungsstrichen" and "Bindestrichen" in German NLP?

Problem In German, the phrase "Haupt- und Nebensatz" has exactly the same meaning as "Hauptsatz und Nebensatz". However, when transforming both phrases using e.g. spacy's ...
user avatar
1 vote
1 answer
76 views

How can I get the output of a Keras LSTM layer?

I want to get the output (that is a vector) of a LSTM layer of a network built in Python using Keras and that is trained to classify sentences (i.e. sequences). How can I do it ? My attempt has been ...
user avatar
2 votes
1 answer
66 views

BERT - The purpose of summing token embedding, positional embedding and segment embedding

I read the implementation of BERT inputs processing (image below). My question is why the author chose to sum up three types of embedding (token embedding, positional embedding and segment embedding)?
user avatar
  • 123
1 vote
2 answers
37 views

Naive Bayes as a baseline model in an NLP task

I want to use the Naive Bayes model as a baseline in an classification task that I am working. I found this really useful tutorial: https://www.geeksforgeeks.org/applying-multinomial-naive-bayes-to-...
user avatar
0 votes
0 answers
3 views

Terminology for meta content inside text documents

My problem is that I want to systematically handle document internal meta-content in NLP processing, but I don't know how to find relevant resources. By meta-content I'm referring to content that ...
user avatar
  • 101
0 votes
0 answers
21 views

Self-supervised learning. Why is it useful?

Self-supervised learning implies that algorithms are trained to predict missing pieces. Say, I take a sentence "I like cats", remove the word "cats" and train an algorithm to ...
user avatar
  • 1
0 votes
0 answers
20 views

how can i average subword embedding?

how can i average subword embedding vectors to generate an approximate vector for the original word as i get the embedding using this function ...
user avatar
  • 15
1 vote
1 answer
24 views

Latent space vs Embedding space | Are they same?

I am going through variational autoencoders and it is mentioned that: continuity (two close points in the latent space should not give two completely different contents once decoded) and completeness ...
user avatar
  • 135
2 votes
3 answers
38 views

Best way to vectorise names and addresses for similarity searching?

I have a large dataset of around 9 million people with names and addresses. Given quirks of the process used to get the data it is highly likely that a person is in the dataset more than once, with ...
user avatar
  • 187
0 votes
0 answers
67 views

KerasClassifier with random search: can't pickle _thread.RLock objects

I created a simple neural network for binary spam/ham text classification using pretrained BERT transformer. Now I want to apply randomized search for tuning the hyperparameters. For now the only ...
user avatar
  • 257
0 votes
1 answer
20 views

Using KerasClassifier for training neural network

I created a simple neural network for binary spam/ham text classification using pretrained BERT transformer. The current pure-keras implementation works fine. I wanted however to plot certain metrics ...
user avatar
  • 257
0 votes
0 answers
14 views

Multi-Label Text Topic Classification

I have a huge dataset of messages/comments classified with topics. The dataset consists of 1kk records and have a total of 90 topics, like this: ...
user avatar
3 votes
1 answer
59 views

Compare Books using book categories list NLP

I have a database of books. Each book has a list of categories that describe the genre/topics of the book (I use Python models). The categories in the list most of the time are composed of 1 to 3 ...
user avatar
0 votes
1 answer
32 views

What happens when the length of input is shorter than length of output in transformer architecture?

Given standard transformer architecture with encoder and decoder. What happens when the input for the encoder is shorter than the expected output from the decoder? The decoder is expecting to receive ...
user avatar
0 votes
0 answers
10 views

create a model to identify the skills priroty

I have created a dataset that has row information as [jobtitle, (skill1, skill1_score), (...), (skilln, skilln_score)]. To create the dataset, I have extracted job titles and job descriptions that ...
user avatar
  • 199
0 votes
0 answers
15 views

LSTM for sentiment analysis

I saw this tensorflow model which is used for telling if text is positive or negative, and I don't fully understand it. I know that LSTM saves the words and predict the next words based on the ...
user avatar
  • 1
0 votes
0 answers
14 views

How to identify the limiting factor in my text classification model?

I am working on building a comments classification model, with about 2500+ comments (varying in length from 5 to ~110 words) and 11 categories (yes I understand the ratio is quite bad). So far, I have ...
user avatar
0 votes
0 answers
12 views

Prune albert model using pytorch

I using albert model to retrain custom corpus with pytorch. I need to prune the model for better performance. Any code source to implement pruning for pretrained models?
user avatar
  • 48
0 votes
1 answer
56 views

How to have a fixed no of features for input layer of a neural network when using TF-IDF

So basically my question is hypothetically lets say: I have a column containing 2000 rows of texts, and when I apply tf-idf, I get 27 features like shown below. Now once I do that, I could consider ...
user avatar
0 votes
0 answers
21 views

What is the effect and How "MATMUL" works for Query and Key in Self Attention in Transformer Encoder architecture?

I am learning about the Transformer Architecture: Attention is all you need by coding it from scratch in pytorch. I got this awesome video on Youtube explaning how ...
user avatar
  • 313
0 votes
1 answer
50 views

Start & End Tokens in LSTM when making predictions

I see examples of LSTM sequence to sequence generation models which use start and end tokens for each sequence. I would like to understand when making predictions with this model, if I'd like to make ...
user avatar
  • 1
0 votes
1 answer
12 views

Customer Voice - Topic Modeling

I am working on a project to classify some messages received from our customers. Basically I have to get the main problem of those messages (hundreds of messages are received every day). Our SAC team ...
user avatar
  • 1
0 votes
3 answers
47 views

Does word2vec fail for window size equal to sentence size

Will word2vec fail if sentences contain only similar words, or in other words, if the window size is equal to the sentence size? I suppose this question boils down to whether word to vec considers ...
user avatar
0 votes
1 answer
14 views

How do companies handle changing natural language

I am assuming large social medias like Twitter handle hashtags using some sort of embedding, so that similar tweets can be found or suggested. Maybe this is a bad assumption- maybe someone can clarify....
user avatar
2 votes
1 answer
276 views

My custom stop-words list using tf-idf

I want to make my own stop words list, I computed tf-idf scores for my terms. Can I consider those words highlighted with red to be stop word? and what should my threshold be for stop words that ...
user avatar
  • 45
0 votes
0 answers
20 views

Finetune XLM-RoBERTa on Tensorflow

I want to finetune pre-trained XLM-RoBERTa from HuggingFace for Text classification. I have categorical data in English. I want to finetune model on Tensorflow-keras. Can anyone let me know how can I ...
user avatar
0 votes
1 answer
36 views

WMT: What are the differences of WMT14, WMT15 and WMT16 datasets?

Each year, the Workshop on Statistical Machine Translation (WMT) holds a conference that focuses on new tasks, papers, and findings in the field of machine translation. Let's say we are talking about ...
user avatar
1 vote
1 answer
65 views

How to prepare texts to BERT/RoBERTa models?

I have an artificial corpus I've built (not a real language) where each document is composed of multiple sentences which again aren't really natural language sentences. I want to train a language ...
user avatar
  • 141
0 votes
0 answers
119 views

Finetune XLM-RoBERTa on TF-keras for text classification

I am trying to finetune pre-trained XLM-RoBERTa on Tensorflow-keras. I am using dataset in English for text classification. I have used xlm-roberta-base tokenizer to tokenize the sentences. I am using ...
user avatar
0 votes
2 answers
75 views

Word Embedding Dimensions Reduction

In my NLP task, I use Glove to get each word embedding, Glove gives 50 float numbers as an embedding for every word in my sentence, my corpus is large, and the resulted model is also large to fit my ...
user avatar
  • 13
0 votes
1 answer
34 views

Why is the accuracy of multi-labels classification using the sigmoid" activation function is much more than using "softmax?

I do multi-label text classification using Bi-LSTM classifier, that means there are instances in the dataset of 11 classes that have more than 1 label. When I use the "sigmoid" activation ...
user avatar
0 votes
1 answer
48 views

How to compute sentence embedding from word2vec model?

I am new to NLP and I'm trying to perform embedding for a clustering problem. I have created the word2vec model using Python's gensim library, but I am wondering ...
user avatar
0 votes
0 answers
27 views

What is the difference between fine-tuning and retraining for the pre-trained word embedding model such as Glove?

I have two questions: What is the difference between fine-tuning and retraining for the pre-trained word embedding model such as Glove, and which is the best for a specific domain? Do I need to ...
user avatar
0 votes
1 answer
168 views

Should we clean text data before applying Vader for getting sentiment

What I meant by data cleaning is that Removing Punctuations Lower Casing Removing Stop words Removing irrelevant symbols, links and emojis According to my knowledge, things like Punctuations, ...
user avatar
0 votes
0 answers
18 views

How to find similar document (using gensim) given two or more other documents?

I am developing a similarity program to compare documents, and I’ve successfully trained my model with Gensim (TFIDF and LSI) in order to compare two documents of each other, and it works great. I can ...
user avatar
  • 101
0 votes
2 answers
24 views

How to evaluate the quality / trustworthiness of textual information?

Suppose I have a corpus of text (which can be used for learning). The text consists of proper names like street names: ...
user avatar
  • 6,872
0 votes
0 answers
14 views

How different are the word embeddings trained from Skipgram and CBOW?

Since what we are interested about usually from CBOW and Skipgram are the by-product word embeddings from the networks, how does the word embeddings they produce differ? When to use which to get the ...
user avatar
  • 359
0 votes
0 answers
13 views

name entity recognition on misspeled words produced by OCR

I need to do entity recognition on a set of text data. There are two important aspects here text data is produced from an OCR which infact has tons of mis-spelled words. For example it produces ...
user avatar
3 votes
0 answers
31 views

Understanding Kneser-Ney Formula for implementation

I am trying to implement this formula in Python $$ \frac{\text{max}(c_{KN}(w^{i}_{i-n+1} - d), 0)}{c_{KN}(w^{i-1}_{i-n+1})} + \lambda(c_{KN}(w^{i-1}_{i-n+1})\mathbb{P}(c_{KN}(w_{i}|w^{i-1}_{i-n+2})$$ ...
user avatar
  • 237
0 votes
0 answers
21 views

Transformer model comparison for binary sentiment classification

On two independent datasets, I am comparing XLNet and BERT models with binary sentiment classification tasks: the Twitter dataset, where sentences are short, and the IMDB review dataset, where ...
user avatar

1 2
3
4 5
47