1
$\begingroup$

I have been testing pre-trained Opus-MT models ported to transformers library for python implementation. Specifically, I am using opus-mt-en-fr for English to French translation. And the tokenizer and translation model is loaded via MarianTokenizer and MarianMTModels--similar to code examples shown here on huggingface. Strangely, for the same pre-trained model translating the same English input on an identical machine, I have observed anywhere between 80+ ms and (whopping) 4 s per translation (example input = "kiwi strawberry").

Wonder if anyone has observed similar behaviours, and what could cause such a wide variation?

$\endgroup$

1 Answer 1

0
$\begingroup$

It could be for various reasons, the first being the words' rarity, then the sequence complexity. Such models work with weight assignments for many words and sequences of words, and the rarest the word, the longer the search.

In addition, GPU/CPU and memory access time could be altered by other running applications in parallel.

To understand what is going on, I suggest testing several sentences from simple to complex, changing current to rare words, and with nothing else running in parallel.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.