0
$\begingroup$

I'm using Python and I want to compare the output of two machine-translation (ish) systems. Most of the tools seem to be focused on sentence-by-sentence evaluation. Either I get memory blow-ups with pyter, or weird results with sacrebleu.

Fundamentally I just want:

$ calculate_ter file1.txt reference.txt

Or

>>> calculate_ter(file1_str, reference_str)

I do not really care specifically about sentence boundaries or line boundaries. I just want to know how different these two texts are.

$\endgroup$
0

1 Answer 1

0
$\begingroup$

Partial answer: I think I can implement calculate_ter like this:

def evaluate_ter(hypothesis, reference):
    """
    Evaluate TER between hypothesis and reference.
    """
    ter_score = TER().corpus_score([hypothesis], [[reference]])
    return ter_score

But it does not scale up to documents of multi-megabytes. Is there a better implementation which does?

$\endgroup$
1
  • $\begingroup$ The only way you could improve this is by checking the implementation of the module that does TER().corpus_score. Maybe some offer greater parallelization than others. But appart from this, there is no easy way to avoid comparing both files. You have to, in order to find the error rate. $\endgroup$
    – liakoyras
    Jun 6, 2023 at 11:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.