New answers tagged nlp
0
First, a tokenizer doesn't have a dictionary of predefined words, so anyway it doesn't make sense to "add a new token" to a tokenizer.
Instead it uses indications in the text in order to separate the tokens. The most common indication is of course a whitespace character " ", but there are lots of cases where it's more complex than that. ...
0
With relative position bias, you are extending the concept of self-attention to also encode the distance between any two tokens. Basically you let the model itself learn the relative distance between any 2 tokens instead of feeding that information yourself. Most of the time (as shown in the paper), the model does a good job at figuring out the relationships ...
0
Ques 1.
You decide how you want your padded pooling layer to behave.This is why pytorch's avg pool (e.g., nn.AvgPool2d) has an optional parameter count_include_pad=True: By default (True) Avg pool will first pad the input and then treat all elements the same.
On the other hand, if you set count_include_pad=False the pooling layer will ignore the padded ...
0
I've tried a lot of automatic methods and data based methods and none really get all of them right. If the word is in the dictionary, its a sure thing to get the right number of syllables. Failing that we can try an automatic method. In your case of Rohit works to say it has 2 syllables. Comes has 1, karate has 3, readier has 3, Siberia has 4, insouciance ...
1
I am currently working on a similar project but limited only to brand detection in product titles, the task is a named entity recognition task and can be solved by different models, the most used ones are BI-LSTM + CRF (Bidirection LSTM with a CRF layer on top). You could try to use spaCy for the task which has a nice documentation and good workflow to train ...
1
About the first piece of code you posted:
At least from the apparent behavior, I would say your code computes the average of all subword vectors in a sentence, not for each word.
To compute word-level representations, you should average only the subwords belonging to a specific word, not all subwords in the sentence.
As a side note, I would suggest not to ...
2
In case you have labeled data (previous complaints labeled by humans), you can implement a standard binary text classification model. A rather simple approach would be to encode the text e.g. as TFIDF or "one hot" and run a simple classification task to learn of some text belongs to label "referred" or "not referred" (which ...
0
What you are looking for is a so-called sentiment-analysis in NLP. In this article, you will find an instruction on how to train a Convolutional Neural Network with a BERT Encoder for a sentiment-analysis.
You could use this example and just replace the training data with movie reviews (positive / negative) with some of your labeled data. Definitely worth a ...
1
The main difference it that BERT includes attention mechanisms, whereas Doc2Vec doesn't.
Attention mechanisms are functions to detect context between words, i.e. learning from words positions using attention weights.
This gives a better result than classic embedding approaches like Doc2Vec, thanks to a contextual approach of data.
On the other hand, BERT can ...
0
K means has certain assumption which leads to high bias due to hard assignments. In case of topic modelling documents may have overlapping classes i.e. a document may have two topics in it.
I would suggest you to try Non Negative Matrix factorisation over K means to check the results.
1
One option is to use a pre-trained embedding space. The pre-trained embedding space will have much lower dimensionality and most likely all of the words in your corpus will be in it.
0
Better approach would definitely be supervised learning model. There are two alternatives for you to go:
(1) What you could try is to use a transformer model that was trained on another sentiment case, like movie or restaurant reviews. First, you could try how this model works for your use-case and then use it to label your unlabeled data.
(2) Or you could ...
0
One option is to embed all the information in a single space. The embedding space would contain the tokens and feature names.
Often times the tokens are changed to track the provenance. For example, science__DOMAIN and professor__COMMENT_BY.
An example of a package that does that is StarSpace.
0
Another option would be to use integrated gradients to get an attribution for each word in a review and add them up over all reviews. Then you know for each word whether it let to a positive or negative review.
This is a practical use case on how to use integrated gradients.
0
You can use this tutorial on a text-based classification with BERT encoder and Convolutional Neural Network. It should work as well with more than two classes.
1
One option could be to use a BERT encoder to tokenize and encode the words and then use a Convolutional Neural Network for the classification task. If you need a tutorial on how to do it, check this article.
Also you can fine-tune a Transformer model, like BERT or Google's T5, to do the classification. But they can take long to train, so try CNN first and if ...
0
There is not such difference between zero padding & character padding ,as we applying padding to extract the edges & gradients to form the object for better learning with respect to human vision.
Even with images mostly people use zero padding which creates black background but depending on the datasets & problem statement padding has to change ...
0
One example is "Self-improving Chatbots based on Reinforcement Learning" by Debmalya Biswas. There is a paper and code.
1
The practice for word2vec is to use only target (hidden) embeddings. But there are some works (for example this paper) about combining target vectors with context vectors. It's not always the case that you will achieve better results by combining embeddings.
In the GloVe paper, the authors achieved a small boost in performance by summing these vectors.
0
You could use Integrated Gradients to see which words have led to a positive/negative sentiment and then aggregate their scores over your whole dataset. Integrated Gradients are an easy and good way to understand neural network inference.
I found an article on how to use integrated gradients for sentiment-analysis.
0
Tf-idf scores are the normalized counts. It might be more useful to use the raw counts to identify potential stopwords.
1
In my opinion this is a very difficult question, and it's not sure that this can be done.
Symbolic methods and statistical methods are hard to combine. In fact, statistical ML methods became mainstream because they could solve most problems better than symbolic methods. This is especially true in NLP: the multiple attempts at rule-based representations of ...
0
Most natural language generation (NLG) systems have the option to input words as a prompt to seed the system.
1
If you are asking how to integrate this, I would leverage existing search technologies such as storing documents in mongo database or using solr indices just to name a few..
If you are asking on the implementation details, take a look on topic modeling, tf-idf, cosine similarity, synonym replacements, k-nearest neighbors to get you started. A lot of these ...
0
There is a subtle but important difference between "semantic role" and "grammatical role" (I think there's a specific term for the latter but I forgot it).
Grammatical role is strictly about syntax. For example in the sentence "John sent a letter to Mary":
"John" is subject
"a letter" is object
"Mary&...
0
Converting to lower case is a historical method to combat data sparsity. The idea is that if you don't have a lot of data, case usually does't matter, so remove the meaningless variable.
But for NER case is an important clue - capital words are more likely to be proper nouns, for example. So you definitely don't want to lower case things.
In general, ...
0
If you have word vectors, the .vectors property uses them to calculate values. Training a model doesn't modify word vectors. It looks like you're just re-using the word vectors from the large English model, which won't contain your special term, so the fix is for you to train your own word vectors and add them to the model.
1
In my opinion there are two main problems with your approach:
The clustering is extremely unlikely to correspond to sentiment, unless the features that you use for clustering are specifically engineered to represent sentiment. In general text clustering tend to group documents by common words, i.e. similar topic. This might lead to different categories of ...
0
You can use the "extractive summarization" method for your problem. This will help you to extract bullet sentences from PDF's.
0
BERT logits will do good for you. What I mean by logits is:
You will train your BERT sentiment analysis model with your data.
While predicting, you will have logits.
For example Positive:0.1231 Negative: 0.7343, so you will give a rating to negative something like 6-7.
Please research for BERT, logits, and sentiment analysis for your problem.
0
My humble guess is using multiple models. Otherwise, you need to construct a new state-of-art model to do this task.
What i mean by multiple models is: You will "milk" model with milk data, "customer" model with customer data. When you wanna make your guess based on milk, you will use "milk" model.
But of course, I don't think ...
3
It appears that your model is failing to generalize.
One option is to increase the amount and quality of the training data.
Other options include large-scale language model specific regularization such as mixout and AUBER.
4
In order to make your model more robust to different wordings, you may try with data augmentation techniques, that is, creating variations of your sentences and adding them to the training set with the same label as the original sentence.
There are frameworks like TextAttack that offer several text augmentation techniques. Another option is using back-...
Top 50 recent answers are included