Questions tagged [nlp]

Natural language processing (NLP) is a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages. As such, NLP is related to the area of human–computer interaction. Many challenges in NLP involve natural language understanding, that is, enabling computers to derive meaning from human or natural language input, and others involve natural language generation.

Filter by
Sorted by
Tagged with
1 vote
1 answer
35 views

Why we need to 'train word2vec' when word2vec itself is said to be 'pretrained'?

I get really confused on why we need to 'train word2vec' when word2vec itself is said to be 'pretrained'? I searched for word2vec pretrained embedding, thinking i can get a mapping table directly ...
user avatar
  • 359
0 votes
0 answers
14 views

How to produce these tensors efficiently/fast?

I would like to produce the following tensor of size (N*N) where the ones (D) appear as follows: ...
user avatar
1 vote
0 answers
9 views

How are the weights defined in a (linear-chain) Conditional Random Field?

Edit: i saw that i mixed up i (in the graph) and t (in the formula), in the following i equivalent to t I am trying to understand the theory behind linear chain Conditional Random Fields. I have now ...
user avatar
  • 11
0 votes
0 answers
12 views

SHAP KernelExplainer AttributeError numpy.ndarray

I've developed a text classifier of the form of python function that can input a np.array of strings (each string is one observation). ...
user avatar
  • 101
2 votes
0 answers
28 views

Perplexed by perplexity

I've seen 2 definitions of the perplexity metric: $PP = 2^{H(p)}$ and $PP = 2^{H(p, q)}$ If I'm understanding correctly, the first one only tells us about how confident the model is about its ...
user avatar
0 votes
0 answers
12 views

How to print 2-ngrams in LimeTextExplainer

I try to explain the importance of a sentence using the following pipeline with LimeTextExplainer from LIME package. ...
user avatar
  • 101
0 votes
0 answers
7 views

Question about computing language modeling loss with multi gpu

When training BERT or GPT or other language model, we use the mean of cross entropy as loss function(don't consider label smoothing). Here B denote for batch size, len denote target length of i-th ...
user avatar
1 vote
0 answers
8 views

Can I use Bert on data subsets and get a compatible representation for the whole dataset?

I need to build an embedding for a massive amount of phrases. I want to use BERT (through the library https://www.sbert.net/). Can I build a partial representation of the data, say encoding 1000 ...
user avatar
0 votes
0 answers
12 views

Best Approach for this Entity Extraction Problem?

Context I have looked endlessly for a similar question to this but I haven't found one so hopefully someone can offer me some insight. I have a task where I'm given a bunch of employees with their ...
user avatar
1 vote
0 answers
29 views

What approach I should take to extract number entity from dataset

I have the training, validation, and test dataset. The first column has store data and the second column has store numbers. I need to develop an entity extractor model which can extract store numbers ...
user avatar
  • 11
0 votes
0 answers
6 views

Atomic tasks from a complex task using NLP

I have a problem statement when I need to find all the tasks that the server had to do based on a complex task. Example, in a 3D modeling scenario, if the model is queried with a complex task such as &...
user avatar
  • 41
1 vote
0 answers
7 views

How to interpret integrated gradients in an NLP toxic text classification use-case?

I am trying to understand how integrated gradients work in the NLP case. Let $F: \mathbb{R}^{n} \rightarrow[0,1]$ a function representing a neural network, $x \in \mathbb{R}^{n}$ an input and $x' \in ...
user avatar
1 vote
1 answer
24 views

NLP Basic input doubt

I actually have a basic doubt in NLP, When we consider traditional models like Decision trees, The feature column order is important, Like first column is fixed with some particular attribute. So If, ...
user avatar
  • 53
1 vote
0 answers
12 views

Extracting information from bills, tax statements, etc: What ML model to use?

I have a bunch of documents such as bank statements, utilities bills, personal expenditure invoices, etc. The document types range is very broad. Some of these files are saved as pictures, others as ...
user avatar
1 vote
0 answers
9 views

Which Pointers from WordNet are Used for Synset in NLTK

I'm trying to create a custom parser for wordnet and hit a roadblock. I see that there are tons of different pointer_symbols and lots of them seem almost like synonyms but not exactly synonyms. I'm ...
user avatar
0 votes
0 answers
5 views

Machine Learning Model for Automating Data Cleaning

I have a large file of unstructured help data as below: Hospital Name, Address and Contact Trinity Hospital (123 Main Street) c/o John Smith Island Health Centre, Director: Benson Lee, 1 Pike Road ...
user avatar
  • 1
1 vote
1 answer
32 views

NLP LSTM input basic doubt

I have a basic doubt with regards to conversion of text to numbers and feeding it to LSTM. I am aware of the different methods such as OneHot, CountVectorizer, TfIDF, Word2vec etc. My doubt is, If we ...
user avatar
  • 53
0 votes
0 answers
23 views

validation accuracy is stuck at 99.44% from the first training epoch while training accuracy and loss are increasing and decreasing respectively

and I don't understand why is this happening is it a problem with the architecture or the training itself I tried slightly different models but with the same problem. its for jigsaw-toxic-comment-...
user avatar
2 votes
1 answer
37 views

Generate paragraphs from given words

I am trying to build a ML model that. will take a list of words and will try to produce sentences with those words, based on a language model on an existing corpus. Example: ...
user avatar
  • 159
2 votes
0 answers
25 views

skip gram vector representation

I am using SVM for sentiment analysis project , I want to use n-skip gram for vector representation because I need to keep the semantic meaning between words , so I need to make vectors that belongs ...
user avatar
1 vote
0 answers
6 views

Can you use both copy mechanism and BPE?

I read to alleviate the problem of Out of Vocabulary (OOV), there are two techniques: BPE Copy mechanism It appears to me they are two orthogonal approaches. Can we combine the two, i.e., we use ...
user avatar
0 votes
0 answers
11 views

How to use label smoothing for single label classification in hugging face models

I am training a binary class classification model using Roberta-xlm large model. I am using training data with hard labels as either ...
user avatar
  • 257
0 votes
0 answers
13 views

How to find syntactic dependencies in text using unsupervised method and context information?

I know there are ready libraries to find syntactic dependencies and besides supervised methods, I have studied some of the unsupervised dependency parsing which uses POS tags and other mathematical ...
user avatar
  • 111
0 votes
0 answers
10 views

Determining whether keywords are a product or not

I'm trying to determine whether a list of keywords are products or not. These lists are categorical, so the products that a list may contain are all related, as to say, maybe a list contains ...
user avatar
  • 1
3 votes
1 answer
24 views

NLP text representation techniques that preserve word order in sentence?

I see people are talking mostly about bag-of-words, td-idf and word embeddings. But these are at word levels. BoW and tf-idf fail to represent word orders, and word embeddings are not meant to ...
user avatar
  • 359
0 votes
1 answer
15 views

Ways to detect negation in Natural Language Processing?

I am studying Natural Language Processing. What could be the ways to detect negation? There are at least two forms of negation that I can think of. I do not like orange juice. I deny that I like ...
user avatar
1 vote
2 answers
33 views

NLP to calculate similarity ratio between sentences of max 5-6 words

Im looking for a relatively simple NLP algo that would help me rate the similarity between two sentences. These sentences usually range between 1-5 words approximately. Context: A user can create as ...
user avatar
1 vote
1 answer
8 views

How to justify logarithmically scaled frequency for tf in tf-idf?

I am studying tf-idf (term frequency - inverse document frequency). The original logic for tf was straightforward: count of term t / number of total terms in the document. However, I came across the ...
user avatar
2 votes
0 answers
16 views

Multi-Class Document Classification with both known and un-known classes

Currently, I am building a multi-class document classifier which has to classify either 3 known classes, namely "Financial Report", "Insurance_Sheet", "Endorsement", and ...
user avatar
1 vote
0 answers
22 views

Method for multi-label category classification

I’m working on a project that involves a Natural Language Processing methodology. I want to classify categories(label) to biomedical news articles (it can be multi-label) (For example, News 1: ...
user avatar
0 votes
0 answers
13 views

Commercial product name classification with product id

I want to give the probability of how the entered name (user 1) for product X matches with the names (historical names from all users) of product X. I have data with the following structure: while ...
user avatar
2 votes
1 answer
28 views

Learning to Rank with Unlabelled Dataset

I have folder of about 60k PDF documents that I would like to learn to rank based on queries to surface the most relevant results. The goal is to surface and rank relevant documents, very much like a ...
user avatar
  • 51
0 votes
1 answer
16 views

Which is the difference between the two Greek BERT models?

I want to use the Greek BERT which can be found here https://huggingface.co/nlpaueb/bert-base-greek-uncased-v1 However I am confused about which model should I use and which are the differences. The ...
user avatar
1 vote
1 answer
38 views

Natural language processing [closed]

I am new to NLP. I converted my JSON file to CSV with the Jupyter notebook. I am unsure how to proceed in pre-processing my data using techniques such as tokenization and lemmatization etc. I ...
user avatar
  • 9
1 vote
1 answer
39 views

spacy multi label classification help

I would like to create a multilabel text classification algorithm using SpaCy text multi label. I am unable to understand the following questions: How to convert the training data to SpaCy format i....
user avatar
0 votes
0 answers
22 views

Is there an exercise solutions manual for "Foundations of Statistical Natural Language Processing"?

I currently re-read Foundations of Statistical Natural Language Processing and enjoyed it, but as there are some hard exercises and I have doubts about my solutions. I would like to know if there is a ...
user avatar
  • 11
1 vote
1 answer
10 views

Advice on movie per topic classification and relation with rating

I would like to extract topics from a set of movie subtitles, and possibly see if there is any relation with the viewer's rating. I have thought about creating a DocumentTermMatrix where each document ...
user avatar
  • 13
0 votes
0 answers
10 views

How to choose between Genism Word2Vec and Keras embedding?

I've seen this post on the difference between keras embedding and word2vec in Genism. It gives me the impression that Word2Vec in Genism is kinda pre-trained word vectors. I wish very much the ...
user avatar
  • 359
0 votes
0 answers
8 views

The output of CBOW, compared to Skipgram

From my undertanding the desired outputs from skipgram is actually the word embedding for a word, as pointed in red in the picture. But how about CBOW? Is the goal of CBOW training also aim at the ...
user avatar
  • 359
1 vote
1 answer
18 views

How do you handle the free-text fields in tabular data in ML/DL?

While we see a number of cases where the input data is only a single text fields (for the X variable) in NLP tasks, e.g. a tweet with a sentiment label being the only numerical field. But how do you ...
user avatar
  • 359
0 votes
0 answers
9 views

Soft-clustering evaluation with multiple labels

I have an article clustering problem ( the articles are encoded with T5 so I technically have vectors) where each one can have multiple topics as labels(the set of labels is unbounded). I did soft ...
user avatar
0 votes
0 answers
9 views

Weighting Sentence Similarity by salience or frequency

It seems like the new standard in text search is sentence or document similarity, using things like BERT sentence embeddings. However, these don't really have a way to consider the salience of ...
user avatar
1 vote
1 answer
47 views

What is the meaning of two embedding layers in a row?

I've noticed in one deep pre-trained textual neural network that there are two embedding layers in the beginning and I don't quite understand why there are two of them. As far as I understand (correct ...
user avatar
  • 157
0 votes
1 answer
251 views

'list' object has no attribute 'lower' TfidfVectorizer

I have a dataframe with two text columns and I converted them to a list. I seperated the train and test data as well. But while making a base model TfidfVectorizer throws me an error of 'list' object ...
user avatar
2 votes
1 answer
81 views

Proof that multihead works better than single head in transformer

According to this post, the purpose of the multihead is to have 'gradient splitting' across heads, which is achieved by random initialization of weight matrices for Q, K and V in each head. But how ...
user avatar
  • 359
0 votes
1 answer
13 views

What makes differences in each head in the multiheaded attention in transformer?

What makes differences in each head in the multiheaded attention in transformer? As they are fed and trained in the exact same way, except the initialization of weights are different for each head to ...
user avatar
  • 359
0 votes
0 answers
17 views

The separate of K and V is redundant in transformer?

imho, I think the separate of K and V is redundant in transformer, as they are basically the same regardless in encoder self-attention, or decoder self-attention, or even the encoder-decoder attention....
user avatar
  • 359
0 votes
1 answer
353 views

ValueError: The first argument to `Layer.call` must always be passed. for k Fold validation

Here is my model ...
user avatar
1 vote
0 answers
40 views

Can pre-trained transformers (I.e., BERT) handle numerical/spatial data

I’m curious to know if pre-trained transformers could handle search queries that include numerical data or make references to spatial relationships. Take an example dataset of a list of restaurants, ...
user avatar
  • 93
0 votes
0 answers
10 views

Learning an embedding for multiple types of text features

I have a problem where my goal is to learn a representation / embedding for various text features. In the initial formulation my dataset looks like the following: ...
user avatar
  • 161

1
2
3 4 5
47