20

The original paper "BLEU: a Method for Automatic Evaluation of Machine Translation" contains a couple of numbers on this: The BLEU metric ranges from 0 to 1. Few translations will attain a score of 1 unless they are identical to a reference translation. For this reason, even a human translator will not necessarily score 1. It is important to note that ...


15

If you want to tackle the problem from another perspective, with an end to end learning, such that you don't specify ahead of time this large pipeline you've mentioned earlier, all you care about is the mapping between sentences and their corresponding SQL queries. Tutorials: How to talk to your database Papers: Seq2SQL: Generating Structured ...


10

I would say that lemmatization is generally the preferred way of reducing related words to a common base. This Quora question is a good resource on the subject: Is it advisable to choose lemmatization over stemming in NLP? The top answer quotes another good resource that motivates why lemmatization is usually better, Stemming and lemmatization, from ...


6

Because of the encoder-decoder structure. The encoder reads the input sequence to construct an embedding representation of the sequence. Terminating the input in an end-of-sequence (EOS) token signals to the encoder that when it receives that input, the output needs to be the finalized embedding. We (normally) don't care about intermediate states of the ...


5

Yes. That should be fine. What you suggest makes sense and should not bias the results. The reasons you give are good ones. For any reasonable classifier, if the value of an attribute is always zero in the training set, that should cause the attribute to be essentially ignored. There is a simple test to let you confirm this. You can try, for each ...


5

The first line of the BERT abstract is We introduce a new language representation model called BERT. The key phrase here is "language representation model". The purpose of BERT and other natural language processing models like Word2Vec is to provide a vector representation of words, so that the vectors can be used as input to neural networks for other ...


5

As you already understand, the vast majority of the data science work is made with rather high level languages such as Python and R. So it's not a matter of prevalence, it's a matter of which part in the big world of data science you want/can do with your skills and your tools. Imho inventing new models requires: strong theoretical background in maths and ...


5

BLEU scores are based on comparing the translation to evaluate against a gold-standard translation. In general the gold-standard translation is the same source sentence translated by a professional translator, so in theory a professional human translation should always receive the maximum score of 1 (BLEU scores are normalized between 0 and 1). However it's ...


4

Your formula is correct for one $w_i$, but if you want to classify a document, you need to compute $P(c | w_1,\ldots,w_N)$. Then you have $$P(c | w_1,\ldots,w_N) = \frac{P(c)\cdot P(w_1,\ldots,w_N|c)}{P(w_1,\ldots,w_N)} = \frac{P(c) \cdot \prod_{i=1}^N P(w_i|c)}{P(w_1,\ldots,w_N)} \neq \prod_{i=1}^NP(c|w_i)$$ where the second equation holds because of the ...


4

NLTK has an excellent step by step guide on everything you need to convert human language to an SQL query using the nltk package in python. It’s rudimentary, but it answers your question.


4

You can use these tips : Should I exclude them for the corpus and from training the model? You can do this if you don't have a lack of data. But I think excluding 500 docs from 30K docs won't make a big difference in training. The model's generalisation power won't be compromised. should I manually translate them (Requesting natives from each ...


4

BERT is a transformer. A transformer is made of several similar layers, stacked on top of each others. Each layer have an input and an output. So the output of the layer n-1 is the input of the layer n. The hidden state you mention is simply the output of each layer. You might want to quickly look into this explanation of the Transformer architecture : ...


3

From the same blog, In this hybrid approach, we encode the subject and previous email by averaging the word embeddings in each field. We then join those averaged embeddings, and feed them to the target sequence RNN-LM at every decoding step. The BoW part of their hybrid approach is to get the general context of the email conversation by averaging the ...


3

Machine learning is inherently data intensive, and typical ML algorithms are massively data-parallel. Therefore, even when developing new algorithms, high-level mathy languages (like Python, R, Octave) can be reasonably fast if you are willing to describe your algorithm in terms of standard operations on matrices and vectors. On the other hand, for deeper ...


3

Yes, this is feasible. One-class classification is a thing, but it is usually used in a context where it is hard or impossible to get negative samples. In your case, I would argue, you can quite easily get tweets that are not about activism, therefore you can render it as a binary classification, because you have data points of two classes or labels: 1 for ...


3

The need for an encoder depends on what your predictions are conditioned on, e.g.: In causal (traditional) language models (LMs), each token is predicted conditioning on the previous tokens. Given that the previous tokens are received by the decoder itself, you don't need an encoder. In Neural Machine Translation (NMT) models, each token of the translation ...


3

I was surfing around at PyTorch's website and found a calculation of perplexity. You can examine how they calculated it as ppl as follows: criterion = nn.CrossEntropyLoss() total_loss = 0. ... for batch, i in enumerate(range(0, train_data.size(0) - 1, bptt)): ... loss = criterion(output.view(-1, ntokens), targets) loss.backward() total_loss +...


3

Some points first: BERT is a word embedding: BERT is both word and sentence embedding. It needs to be taken into account that BERT is taking the sequence of words in a sentence into account which gives you a richer embedding of words in a context but in classic embeddings (yes, after BERT we can call others "classic"!) you mostly deal with neighborhood i.e. ...


3

It is not uncommon that we can make sense of a sentence without reading it completely. Or when you are having a quick look at a document, you tend to oversee some words and still understand the main point. This is the intuition behind the word dropout. Generally this is done by randomly dropping each word in a sequence following for example a Bernoulli ...


2

BIO(L) tagging is important (but as you correctly noted, not necessary) part of a NER pipeline. Main idea behind such split is to facilitate learning in following manner. Take English as an example, some words will (most likely) never end a Named Entity, like adjectives, so the model will never tag them as the L(ast) part of a named entity. The same applies ...


2

Starting with a small chunk of documents (~50) as a training set with a set of rules implements would be a good starting point. Train using a few algorithms and see if your training accuracy is coming out as acceptable or better than the human accuracy. Then slowly increase your training batch size and see the impact on the results. Once you believe that ...


2

That's a good question. The answer is yes and no. No because the input layer of the CBOW model expects a fixed number of words. So you'll either always input 3 words or 9 words. Yes because you can however set the sequence length as 9 words and provide just 3 words as context while the remaining 6 words can just be zero vectors. Remember in a CBOW each ...


2

This paper might be useful.... https://arxiv.org/abs/1801.06146


2

To complement Fadi's answer, the following are other useful papers on NL to SQL methods. The major difference of these methods is that they support queries that should be answered using more than one table (joining different tables), however the Salesforce paper (and their dataset) is focused on queries on one table at a time. Learning a Semantic Neural ...


2

The difference is not related to the length of the named entities. Rather, it deals with how two adjacent named entities of the same type are labeled. In IOB1 (IOB), B- is only used to separate two adjacent entities of the same type: Today O Alice I-PER Bob B-PER and O I O # or I-PER if pronominals are being tagged ate O ...


2

There are multiple techniques that help with the problem you sketch, the applicability of which usually depends on the classification technique and the corpus. But I get the idea you would be helped by some practical examples. So let's go over some of them. Feel free to comment if I tread over familiar grounds, or you want me to elaborate on some of them. Or ...


2

You could go with different approaches: Lemme point few, You could just extract the keywords or tokenize out of the query using libraries like spacy or nltk, both support german languages. Then go for like page ranking approach, based on query optimization: Uber. You could go for Attention based seq2seq model, where you feed the inputs as questions and the ...


2

NLTK A great starting point for keyword extraction is the NLTK (natural language toolkit) library. To extract keywords, you probably need to tokenize your data, breaking each word out into a token, and ignore the most common or unimportant words known as "stopwords". Assuming you're searching for keywords across a large number of query results, identify ...


2

Welcome to the site. I'm a little disturbed by the other two answers you received here. It sounds like you are skipping a whole lot of steps and wanting to jump right into modeling - that's a massive mistake! You are a scientist! Your role is to create the most fair, unbiased environment possible to let the data speak to you (not the other way around!). ...


2

According to [Caccia et al., 2018], in general textual GANs are no rival for LMs regarding several quality measures. These are the conclusions of the paper: This research demonstrates that well-adjusted language models are a remarkably strong baseline and that temperature sweeping can provide a very clear characterization of model performance. A well-...


Only top voted, non community-wiki answers of a minimum length are eligible