1
$\begingroup$

I have a dataset which looks like this, I have built LSTM model to perform seq prediction

X                   y
1,2                3,6,1,6
2,3                4,9,3,7
3,45               23,4,1,11

This is a sequential dataset with multiple input and multiple outputs, I am not sure how to measure the performance of my model for test data? I tried using accuracy but it is not possible to calculate accuracy for multiple outputs (got error). I tried doing validation loss, but I want to know if there is any better way to calculate performance metric of my model?

$\endgroup$

1 Answer 1

0
$\begingroup$

It should be emphasized that an evaluation measure should always be selected specifically for the target task, not only based on technical characteristics (e.g. sequential). For example one would not necessarily use the same evaluation measure for predicting the top results of a sport event and detecting named entities, even though both tasks are sequential in nature.

This being said, a typical approach for this kind of output would be to measure the similarity/distance between the gold-standard output and the predicted one. A standard way to compare sequences is the Levenshtein edit distance.

$\endgroup$
2
  • $\begingroup$ Thanks for the detailed explanation, does this hold true for the example I mentioned ? The sequences are numbers and not strings $\endgroup$ Commented Aug 27, 2022 at 16:04
  • $\begingroup$ @AbhishekPatil even though it's known mostly for text, Levenshtein is actually a distance measure over sequences of any kind of symbols: words.letters, numbers. For numbers you could probably use a variant which to take into account the difference between 2 continuous numbers. Also there are probably other ways to calculate a distance between 2 sequences of numbers. $\endgroup$
    – Erwan
    Commented Aug 29, 2022 at 13:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.