2
$\begingroup$

I am currently training a transformer model on text data. Is it a good practise to open abbreviations/acronyms in the text data? I did not dins any tips or recommendations about it on internet.

$\endgroup$

2 Answers 2

1
$\begingroup$

In general, it is usual not to do any preprocessing of the text. This, however, depends heavily on your specific case. I would suggest not doing any preprocessing but evaluating the results on the aspects that are important to you and, if you detect any problem that may be solved with some preprocessing step, train a new model with it and compare it with the first model.

$\endgroup$
0
$\begingroup$

It depends on what you want the model to learn and the size of the dataset.

If it is a generative model and you want the model to generative abbreviations/acronyms, then those have to be in the data.

Historically, abbreviations/acronyms were replaced with a standardized version. This increased the signal-to-noise ratio by decreasing the size of the vocabulary and increasing the frequency of each token. This is less of an issue if you are training on very large datasets.

$\endgroup$
1
  • $\begingroup$ it is nlu model, it returns embeddings. i want to do domain adaptation of the model. my train data has about 90k rows $\endgroup$
    – Ir8_mind
    Mar 2, 2023 at 13:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.