Hot answers tagged

4

In order to make your model more robust to different wordings, you may try with data augmentation techniques, that is, creating variations of your sentences and adding them to the training set with the same label as the original sentence. There are frameworks like TextAttack that offer several text augmentation techniques. Another option is using back-...


3

It appears that your model is failing to generalize. One option is to increase the amount and quality of the training data. Other options include large-scale language model specific regularization such as mixout and AUBER.


1

The practice for word2vec is to use only target (hidden) embeddings. But there are some works (for example this paper) about combining target vectors with context vectors. It's not always the case that you will achieve better results by combining embeddings. In the GloVe paper, the authors achieved a small boost in performance by summing these vectors.


1

About the first piece of code you posted: At least from the apparent behavior, I would say your code computes the average of all subword vectors in a sentence, not for each word. To compute word-level representations, you should average only the subwords belonging to a specific word, not all subwords in the sentence. As a side note, I would suggest not to ...


1

I am currently working on a similar project but limited only to brand detection in product titles, the task is a named entity recognition task and can be solved by different models, the most used ones are BI-LSTM + CRF (Bidirection LSTM with a CRF layer on top). You could try to use spaCy for the task which has a nice documentation and good workflow to train ...


1

The main difference it that BERT includes attention mechanisms, whereas Doc2Vec doesn't. Attention mechanisms are functions to detect context between words, i.e. learning from words positions using attention weights. This gives a better result than classic embedding approaches like Doc2Vec, thanks to a contextual approach of data. On the other hand, BERT can ...


1

In case you have labeled data (previous complaints labeled by humans), you can implement a standard binary text classification model. A rather simple approach would be to encode the text e.g. as TFIDF or "one hot" and run a simple classification task to learn of some text belongs to label "referred" or "not referred" (which ...


1

One option is to use a pre-trained embedding space. The pre-trained embedding space will have much lower dimensionality and most likely all of the words in your corpus will be in it.


1

One option could be to use a BERT encoder to tokenize and encode the words and then use a Convolutional Neural Network for the classification task. If you need a tutorial on how to do it, check this article. Also you can fine-tune a Transformer model, like BERT or Google's T5, to do the classification. But they can take long to train, so try CNN first and if ...


1

If you are asking how to integrate this, I would leverage existing search technologies such as storing documents in mongo database or using solr indices just to name a few.. If you are asking on the implementation details, take a look on topic modeling, tf-idf, cosine similarity, synonym replacements, k-nearest neighbors to get you started. A lot of these ...


1

In my opinion this is a very difficult question, and it's not sure that this can be done. Symbolic methods and statistical methods are hard to combine. In fact, statistical ML methods became mainstream because they could solve most problems better than symbolic methods. This is especially true in NLP: the multiple attempts at rule-based representations of ...


1

In my opinion there are two main problems with your approach: The clustering is extremely unlikely to correspond to sentiment, unless the features that you use for clustering are specifically engineered to represent sentiment. In general text clustering tend to group documents by common words, i.e. similar topic. This might lead to different categories of ...


Only top voted, non community-wiki answers of a minimum length are eligible