I have a question as to how the F1 should be calculated in NLP and whether the text normalization is optional or not.

So I have been working on a project where we created a closed-domain extractive QA dataset from scratch, and we are trying to finetune and assess the performance of several LLMs in this new dataset. I came across different definitions of the F1 score, sometimes with text normalization (like in here) and sometimes not. I have run all my experiments without the normalization step for both the EM and F1 scores. Should I rerun all experiments?

Is the



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.