There is already an ML engine that does these extractions, here is a general process layout:
Hre is the orginal paper describing the architecture, features, approaches etc. read it up, instead of me copying it here. cloudscan
Found the solution, with the .classes_ attribute you can find the predicted classes. Then you have to concatenate the predict_proba together with the .classes_.
In Python it looks like this
message4 = "Jeans model Ornella met iets kortere pijpen Van"
bow4 = bow_transformer.transform([message4])
tfidf4 = tfidf_transformer.transform(bow4)
counter = 0
I was surfing around at PyTorch's website and found a calculation of perplexity. You can examine how they calculated it as ppl as follows:
criterion = nn.CrossEntropyLoss()
total_loss = 0.
for batch, i in enumerate(range(0, train_data.size(0) - 1, bptt)):
loss = criterion(output.view(-1, ntokens), targets)
The need for an encoder depends on what your predictions are conditioned on, e.g.:
In causal (traditional) language models (LMs), each token is predicted conditioning on the previous tokens. Given that the previous tokens are received by the decoder itself, you don't need an encoder.
In Neural Machine Translation (NMT) models, each token of the translation ...