# Tag Info

2

If you want a DL approach, I recommend substituting the tf-idf by some kind of word embeddings. For instance, you can take a pre-trained word embedding model, like glove, and average its outputs both in resume and job description, and then compute cosine similarity. However, I recommend to use a contextual word embedding (BERT-like), as the terms in resumes ...

2

Both approaches have been used for sentiment analysis in the literature. From a quick search, we can find these results: Constituency parsing: Deep Recursive Neural Networks for Compositionality in Language [NeurIPS'14] Less Grammar, More Features[ACL'14] A Statistical Parsing Framework for Sentiment Classification [Computational Linguistics journal'15] ...

2

What you need is simply a language model. This is a very common task so you should be able to find code and data easily. This question gives some pointers for Python (be careful, the accepted answer is incorrect according to the two other answers). Applying the language model to a sentence gives you a probability (or a perplexity score, which works the ...

2

I don’t know why your question was voted down! Fuzzy matching is such a common challenge. The best approach I’ve seen is this one: https://towardsdatascience.com/fuzzy-matching-at-scale-84f2bfd0c536 It gives similar results to something like Levenshtein distance but it’s much faster. If you augment the matching approach with hand-coded regex features to spot ...

1

The vocab file contains a mapping from vocabulary strings and indices used for embedding lookup in the model. The merges say how to split the input string into subword units. The algorithm is as follows: At the beginning of merging, a word split into characters and then you greedily search for neighboring symbols that can be merged (i.e., are in a list of ...

1

You can do that. I propose the simplest one conditioned on the fact that number of data is not very large. In case you need more ideas please drop a comment. In this case, you can use the idea of similarity encoding based on Fuzzy String Matching and get the spectral embedding. The amount of data is crucial here as you need to do order of $n^2$ comparisons ...

1

So the question is asking why the first two principal components of your encoded text data is encapsulating all of the variation in the data. One potential issue could be the averaging over word vectors. Suppose for a particular feature across word vectors for a particular post f, there could be an array of positive and negative values. When we then apply an ...

1

GPT-2 does not use a word-level vocabulary but a subword-level vocabulary, specifically byte-pair encoding (BPE). This means that it does not predict the next word but the next subword token. BPE tries to find the pieces of words that are most reusable. BPE also keeps character subwords (e.g. "a", "W"). The subword vocabulary used by GPT-...

1

TypeError: unhashable type: 'list'. You will get this error when you are trying to put list as key in dictionary or set because list is unhashable object. Example you trying to input code such as dict1 ={ 1:'one', [2]:'two'} print(dict1) O/p: TypeError Traceback (most recent call last) in ----> 1 dict1 ={ 1:'one', [2]:'two'} 2 print(dict1) TypeError: ...

1

You need to create a vocabulary of the n-grams, i.e., a numbered inventory of bigrams that you are going to use as features. Typically, these are the most frequent ones. When you create the feature vector, you start with a zero vector and put one (or add one) if the n-gram with the corresponding index appears is in your sentence. Machine learning libraries ...

1

So, your task is to detect the passive voice from sentences. Currently, you have defined some rules to detecting the passive voice and you have noticed that there some exceptions to your defined rules. Therefore, it would be a good idea to develop a model to predict the probability of a sentence being passive (or active). You can do this by encoding the ...

1

According to recent publications, it is not impossible to get BLEU scores as high as yours for English→Irish. Nevertheless, without any other knowledge, they certainly seem too high. From the command line arguments, there does not seem to be any evident problem. The most probable explanation is, as you already pointed out, a data leakage between validation/...

1

BM25 is usually used in information retrieval. In this task, you have a query and a lot of documents(maybe millions), and then you want to find a subset of these documents that are most relevant to your query. A ranking of a set of documents will be provided from the most relevant to the least. If by efficient you mean fast in a computational way. I would ...

1

A quick experiment you can do is to once do the preprocessing steps that you usually do and then feed it to the model and get the results. And once feed the dataset as it is to the model to compare the difference. In my experience doing the preprocessing won't make any difference, based on the dataset it gave me 1 more or less percent difference in accuracy (...

1

When you concatenate, you have to define a priori the size of each vector to be concatenated. This means that, if we were to concatenate the token embedding and the positional embedding, we would have to define two dimensionalities, $d_t$ for the token and $d_p$ for the position, with the total dimensionality $d = d_t + d_p$, so $d>d_t$ and $d>d_p$. We ...

1

So the question is about why positional embeddings are directly added to word embeddings instead of concatenated. This is a particularly interesting question. To answer this question, I will need to firstly separate the differences between sequential networks like RNNs and Transformers, which then introduces this problem nicely. In RNNs, we feed in data (let'...

Only top voted, non community-wiki answers of a minimum length are eligible