Questions tagged [nlp]

Natural language processing (NLP) is a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages. As such, NLP is related to the area of human–computer interaction. Many challenges in NLP involve natural language understanding, that is, enabling computers to derive meaning from human or natural language input, and others involve natural language generation.

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

How to calculate the sparseness of the trigram model?

A corpus contains 1000000 word tokens, 15000 word types, 300000 distinct biagrams and 400000 distinct trigrams. How to calculate the sparseness of the trigram model? (ie calculate the percentage of ...
0
votes
1answer
13 views

GradientBoostingRegressor Text Classifier

I am working to build a text classifier using a Boosting method from sklearn. It is performing quite well, at around 97% accuracy on my test data. However, the problem I am seeing is that if I input ...
0
votes
1answer
13 views

Calculating confidence score in NER

I am working on a problem on Named Entity Recognition. Given a text, my model is detecting the Named Entities and extracting that info for the end user. Now the ask is end user needs a confidence ...
0
votes
0answers
12 views

Recommender System Approaches

I have a 4 datasets with user features, item features, user-item rating and User-item link data. I'm trying to build a recommender system to recommend top 10 items to the user by maximizing NDCG as ...
0
votes
0answers
11 views

What model should i use to extract relation between words

I want to create a ML model which would give a score from 0 to 1 which would signify the relation between them. I know about Relationship Extraction(RE) but that's more related with sentences based ...
0
votes
1answer
19 views

Is this a tried alternative to word embedding for NLP?

I'm searching for research related to my idea, but apparently cannot articulate it well enough to the search engines to show me what's been published on this. My idea: in a deep learning context (text ...
1
vote
0answers
12 views

A simple attention based text prediction model from scratch using pytorch

I first asked this question in codereview SE but a user recommended to post this here instead. I have created a simple self attention based text prediction model using pytorch. The attention formula ...
0
votes
1answer
18 views

Currency Normalization for Salary Prediction

I have a dataset (350k data points) with data of employees across different regions over the last 10 years. The dataset consists of their skills, the region they are in, the industry, their current ...
0
votes
0answers
10 views

Is there any TF implementation of the Original BERT other than Google and HuggingFace

Trying to find any Tensorflow/Keras implementation of the original BERT model trained using MLM/NSP. The official google and ...
0
votes
0answers
16 views

How to extract important text markers from samples to identify patterns?

Problem I have collected a decently large set of movie trailer titles from various Youtube trailer channels. I'd like to extract or infer the movie title and release year from this set in some way to ...
1
vote
0answers
23 views

What is the learning path for the role of NLP engineer a beginner should follow? [closed]

I am data science enthusiast and have interest in NLP. How should I develop my understanding for the domain and prepare for the role of an NLP engineer? What are the must have skills to master NLP? ...
0
votes
0answers
22 views

Effective way to find similarity between utterance(short text) and question(long text)

The challenge I have is a bunch of questions(long text) that are closely matching with an user utterance (short text). I have tried cosine similarity & Tf-iDF, BM25,Jaccard similarity, etc., but ...
2
votes
0answers
20 views

Evaluate Topic Modelling on synthetic data

I try to find the optimal number of topics on a synthetic corpus (so a list of lists of tokens I generate using various parameters). I, therefore, know the true number of topics and the true topics ...
0
votes
2answers
21 views

BERT Self-Attention layer

I am trying to use the first individual BertSelfAttention layer for the BERT-base model, but the model I am loading from torch.hub seems to be different then the ...
1
vote
1answer
15 views

Human readable format for clusters of word vectors

Let's say I have pretrained word2vec model and apply it to dataset consisting of article titles from "The Guardian". It seems pretty obvious that titles coming from "Science" ...
0
votes
1answer
23 views

BERT embedding layer

I am trying to figure how the embedding layer works for the pretrained BERT-base model. I am using pytorch and trying to dissect the following model: ...
2
votes
1answer
36 views

Training a model purely on weak labels

I have read a couple of papers now use rules-based system to create weak labels and then train a BERT-based model only using these weak labels. Both studies have reported better performances on ...
0
votes
0answers
15 views

How to design optimizer for combined model training in Pytorch

I am trying to train a embedder. So, I have an architecture for the model to embed texts. And I have another model architecture that will take the inputs from the output of the first model and predict ...
0
votes
0answers
20 views

Paper/Idea to get the most attractive chapter in a novel

I face a problem as below: There is a novel app and need to find the most attractive chapter in previous 30 chapters of each novel So that we can limit a user continue unless he make payment I didn'...
0
votes
1answer
40 views

Any research on relationship between the dimensions of a (word2Vec) space and how the human mind constructs meaning (or reality) through language?

Neuroscience is still trying to "find" how the mind (and language) somehow "works". Is there any theory linking a (low-dimensionality) embedding space (like word2Vec) to a mind (...
0
votes
0answers
7 views

Does ktrain's performance the same as in using Pytorch?

I am new to natural language processing (NLP) that TensorFlow and Pytorch are too hard for me. Thanks to the technology, there's another choice to build the model: ktrain. But, when I went through the ...
0
votes
0answers
15 views

Identifying the role of an entity (e.g. a company) in a sentence

I have a general question about how to distinguish between the following two types of sentences: Type 1: ...
0
votes
1answer
31 views

Pytorch: understanding the purpose of each argument in the forward function of nn.TransformerDecoder

According to https://pytorch.org/docs/stable/generated/torch.nn.TransformerDecoder.html, the forward function of nn.TransformerDecoder contemplates the following arguments: tgt – the sequence to the ...
1
vote
1answer
19 views

How do I split contents in a text that would include two or more different themes (context) in NLP?

For example, a text: "The airlines have affected by Corona since march 2020 a crime has been detected in Noia village this morning" the output should be: The airline companies have affected ...
1
vote
1answer
30 views

Masked Language Modeling on Domain-specific Data

My goal is to have a language model that understands the relationships between words and can fill the masks in a sentence related to a specific domain. At first, I thought about pretraining or even ...
0
votes
0answers
15 views

complete entity extraction from unstructured data

I understand there are many techniques/libraries/packages to extract named entities like people, places etc. from data. Personally, for me an entity is something like: ...
0
votes
1answer
31 views

Combining textual and numeric features into pre-trained Transformer BERT

I have a dataset with 3 columns: Text Meta-data (intending to extract features from it, then use those i.e., numerical features) Target label Question 1: How can I use a pre-trained BERT instance on ...
0
votes
1answer
20 views

Embedding from Transformer-based model from paragraph or documnet (like Doc2Vec)

I have a set of data that contains the different lengths of sequences. On average the sequence length is 600. The dataset is like this: ...
1
vote
0answers
39 views

BERT MLM overfitting [closed]

We are training the BERT model on masked language modeling task for the Russian Language. Our dataset consists of 60 mln texts with (128 tokens for each text) from online social networks, ...
1
vote
2answers
25 views

How to decide to go with BOW or TFIDF

I know that there are methods that help in selecting features such as Matual Info, and Info Gain, etc. But for datasets with thousands of records and thousands of features it is time consuming to ...
0
votes
3answers
32 views

Best Python NLP library for supervised topic classification

I have a labeled dataset that I have ingested into a dataframe. It consists of news articles, ...
2
votes
3answers
68 views

Word2vec outperforming BERT, possible?

I'm trying to solve a multilabel classification (dataset is tweet text) using a combination of BERT and CNN. As a benchmark, I'd compare it to other word embeddings, one of which is Word2vec. After ...
4
votes
1answer
75 views

Calculating optimal number of topics for topic modeling (LDA)

am going to do topic modeling via LDA. I run my commands to see the optimal number of topics. The output was as follows: It is a bit different from any other plots that I have ever seen. Do you think ...
0
votes
0answers
10 views

Doing feature extraction from name data?

I'm working with a genre prediction application right now, and I was curious about handling name data. I was planning to try to use that in the prediction(as a normal human can usually estimate ...
0
votes
1answer
18 views

Training Objective of language model for GPT3

On page 34 of OpenAI's GPT-3, there is a sentence demonstrating the limitation of objective function: Our current objective weights every token equally and lacks a notion of what is most important to ...
1
vote
1answer
19 views

Naives Bayes Text Classifier Confidence Score

I am experimenting with building a text classifier using Naive Bayes which has been pretty successful on my test data. One thing i am looking to incorporate is handling text that does not fit into any ...
0
votes
0answers
21 views

How to classify all words in a sentence with a context?

I have the names of the companies (in Russian). The name can contain abbreviations, words with capital letters, words with lowercase letters, and mixed words. The model is trained according to the ...
0
votes
0answers
16 views

How to filter data samples which do not improve classifier?

I have a text dataset with noisy labels and an unbalanced shape. There are various ways to find features which do not drive improvement in some metric, and help to prune those from the pipeline. I ...
1
vote
0answers
17 views

Extracting indirect quotations

I found textacy Python library, built on top of Spacy, a useful tool to experiment with direct quotation extraction. Unfortunately, ...
0
votes
1answer
14 views

What is the common practice for NLP or text mining for non-English?

A lot of natural language processing tools are pre-trained with corpus in English. What if ones need to analyze, say, Dutch text? The blogs I find online are mostly saying traslating text into English ...
0
votes
2answers
52 views

How to JUST represent words as embeddings by pretrained BERT?

I don't have enough data (i.e. I don't have enough texts) --- have only around 4k words in my dictionary. I need to compare given words, then I need to representate it as embedding. After the ...
1
vote
1answer
21 views

Should we also include negative instance in cross-validation process of one-class classifiers?

For a one-class classifier to do text classification, only positive instances are used for training. However, in the cross-validation process to select the best hyperparameters, should we also include ...
0
votes
1answer
19 views

Addressing polysemy in NLP tasks

Looking for modern algorithms using NN Language Model implementations addressing polysemy in NLP tasks, including text classification, question answering and topic modeling. Transfer/Zero-short ...
0
votes
1answer
28 views

How to cluster words automatically?

I have a problem where I have a list of n words with truly k different ones (k is unknown) because some may be malformed or contracted. I would like to automatically cluster them. I thought about ...
-1
votes
1answer
24 views

Problem with mord library function 'Logistic AT'

I wanted to run a an ordinal logistic regression on my bag of words. I have used the code below with logistic regression but now I have modified it for an ordinal logistic regression. However, when I ...
2
votes
3answers
162 views

The reason behind using a pre-trained model?

These last month I have been studying all about word embeddings and the most known pre-trained word embeddings, Word2Vec, GloVe, FastText, etc. I have read many times how important It is to take ...
0
votes
0answers
24 views

Having trouble reducing MSE error for SVR model sklearn

I'm trying to create an SVR model to predict the number of comments a headline will receive for the following dataset : https://www.kaggle.com/benjaminawd/new-york-times-articles-comments-2020?select=...
0
votes
0answers
21 views

Extracting “hidden” costs from financial statements using NLP

I'm designing a NLP model to extract various kinds of "hidden" expenses from 10-K and 10-Q financial statements. I've come up with about 7 different expense categories (restructuring costs, ...
1
vote
1answer
16 views

NLP methods specific to a language?

What NLP methods / algorithms depend on the features existing only in some languages? For example, does French has any NLP algorithms that English NLP and Spanish NLP do not have?
0
votes
1answer
22 views

Word2Vec: Identifying many-to-one relationships between words

Standard introductory examples in Word2Vec, like king - queen = man - woman and tokyo - japan = london - uk, involve one-to-one ...

1
2 3 4 5
38