0
$\begingroup$

A lot of natural language processing tools are pre-trained with corpus in English. What if ones need to analyze, say, Dutch text? The blogs I find online are mostly saying traslating text into English as pro-processing. Is this the common practice? If not, then what? Also, does how similar a language is to English have an impact on the model performance?

For some also widely speaking languages (e.g French, Spanish), do people construct corpus in their own language and train models on it? Forgive my ignorance because I'm not able to read papers in many languages.

$\endgroup$
1
$\begingroup$

Translation as a pre-processing step is usually sufficient for many tasks (e.g. sentiment classification), but naturally undesirable for other tasks e.g. grading someone in written Dutch fluency.

Hence, for these tasks, the objective is:

  • Be able to train a language model for your specific language
  • However, you want to be able to do this with minimal resources

Thus, people in research view this as few-shot learning. One of the most common approaches to this is to exploit meta-learning, i.e. use models that have been pre-trained using many tasks (languages in this case) and they have been trained so that only a few gradient steps and little data is required for them to be applicable to a new task (Dutch in your example). An example of one such model you could finetune (transfer learning) is called multiliingual BERT... this was back in 2019, so I expect since many other multilingual language models have emerged - these can be good initialisation points for any bespoke model you want to train for some new language...

$\endgroup$
1
  • $\begingroup$ Heel interessant! I'm doing a project with BioBERT right now. Will check multilingual BERT! Dank u wel! $\endgroup$ – Paw in Data Apr 14 at 23:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.