Hot answers tagged

4

The restriction in the maximum length of the transformer input is due to the needed amount of memory to compute the self-attention over it. The amount of memory needed by the self-attention in the Transformer is quadratic on the length of the input. This means that increasing the maximum length of the input, increases drastically the needed memory for self-...


2

You'll probably find pointers in the literature searching for "offensive text detection". There are many variants/overlaps with related tasks such as detecting bullying. There are probably also annotated datasets around, in case you want to use these as training data.


2

One way to approach it is to split the sentence into tokens and count the number of tokens that are profanities. import re def tokenize(text): return re.findall(r'\w+', text.lower()) profane_tokens = {"nerfherder"} sentence = "Why you stuck-up, half-witted, scruffy-looking nerfherder!" tokens = tokenize(sentence) # Rate: number of occurrences ...


2

This problem is multiple choice answering question. I can see you have already tried gensim, doc2vec etc. You can try pytorch based transformer solution. Here is the link: multiple-choice . You can create your data in swag format and remove --do_train in below code for prediction on your dataset. It has been trained on swag dataset and has given decent ...


1

You're doing a big mistake in your code, which is applying the vectoriser before the train/test splitting. The vectoriser should be fit only on the training dataset, then the learned counts should be applied to the test set. Instead you applied the vectoriser to the whole data which you then splitter into train and test. # THIS IS OK # train_data = np....


1

For future readers: as Ben wrote in the comments, it is really hard in general that len(my_dict) != len(my_dict.items()). When these kind of strage behaviours happen, it is always a good practise to perform some routine checks: Clean your environment from every variable, even better restart the kernel and then run again your code. Check your code for ...


1

Question 1: To do so, I would use the Gensim wrapper of FastText because Gensim has a predict_output_word which does exactly what you want. Given a list of context words, it provides the most fitting words. Question 2: It is up to the user. FastText isn't inherently CBOW or Skipgram. See: https://fasttext.cc/docs/en/python-module.html Question 3: Yes, ...


1

I think the task you're looking for is called "Punctuation restoration" or "Punctuation prediction". An example of this is described in this paper, or this other paper. Unfortunately I am not aware of available per-trained model that you might apply to your dataset. Honestly I think that it is unlikely that something trained specifically for costumer service ...


1

There are several ways you can obtain document embeddings. If you want to obtain a vector of a document that is not part of the trained doc2vec model, gensim provides a method called infer_vector which allows to you map embeddings. You can also use bert-as-service to generate sentence level embeddings. I would recommend using Google's Universal Sentence ...


1

Firstly, fixing the input size of a model is more of an architectural decision than a problem. By fixing input size we: Limit the GPU memory usage while training Reduce the training time per epoch Reduce evaluation time per input sample If you want to, you can train your own transformer model by increasing the size of the input or increasing the number of ...


1

If I got your problem description correct, you are looking for a recommender system like for example used by Netflix or Amazon. State of the art solution would be to use Latent Dirichlet Allocation topic modeling to make recommendations based on topics (in your case, topics would be the tags). Here is a very good video tutorial on this topic: https://youtu....


1

One solution could be to train a single embedding space, StarSpace is one such implementation. That single embedding space would contain all users, documents, and tags. Then it is a nearest neighbor search to recommend any combination. Given a user, find the nearest documents. Given a tag, find the nearest tags … For new entities (i.e., users, documents, or ...


1

There's a lot of ways to do this, one approach is to use token-based matching. You can use this to easily find any "tokens" in the text, like names, places, or just plain words. Methodology I'd recommend using Rule-based Entity Recognition in spaCy. You'll define the "rules" of what the entity looks like, here's the example from the docs where we define ...


1

An automatic system for finding synonyms using word embeddings is not possible. Word embeddings find co-occurrence. For example, "good" and "bad" co-occur together in a corpus, thus are near each other in the embedding space. However, "good" and "bad" are antonyms. A copilot system could work. Word embeddings can find a set of candidate synonyms by finding ...


1

I'm not sure that building a model using CRF is the best approach here. It's going to require quite a bit of training data and effort to get it working like you want. Dates are pretty structured in most cases so there are more straightforward approaches to extracting them. Stanford's SUTime library, for instance, does exactly what you're describing in your ...


1

One idea that could be leveraged - use all the model answers and learning material, find some sort of semantic similarity between all that text and the student answer. The higher the similarity with a model answer with a specific grade tag, the higher the probability of assigning that answer that particular grade. You can couple this with a metric of ...


1

Limit outputs od decoder to N. Not sure how easy it would be, probably a bit digging into official implementation but after that the main "skeleton" of the GPT2 is usable, meaning that all of the pre-training can be reused to produce meaningful sentences.


1

The problem is not really "new text", since by definition any classification model for text is meant to be applied to some new text. The problem is out of vocabulary words (OOV): the model will not be able to represent words that it didn't see in the training data. The most simple way (and probably the most standard way) to deal with OOV in the test data is ...


1

This is an NER problem. Rather than you splitting your sentence to words and finding the right word from dict, I suggest you use an NER (may be spacy NER mentioned by @jindrich). This NER will point out right block of information from the text your sentences. Once you get an Entity then you can parse its value. If it is quantitative then it is easy to ...


1

You have to give your training set to the model to be trained _= pipe.fit(triningSet.data, triningSet.target) I don't see any training dataset here. you have to fit the CountVectoriser to your data set.


Only top voted, non community-wiki answers of a minimum length are eligible