Can bert uncased predict text classification on foreign data?

Question

I am trying to do the fake news/real news classification and used a pre-trained bert uncased model as transfer learning and it gave a solid 81% accuracy. But the problem is while doing sanity checks, I found my dataset has some Korean/Chinese text articles and these are some real news and it gave the trustworthy score(basically probability) as 60-70%. If Bert-uncased is only for the English language, I'm just thinking about how it processes those languages. Does anyone have any insights?

Answer 1

BERT is a pre-trained model that is trained on a large amount of text data in multiple languages. While the BERT-uncased model is primarily trained on English text, it also has some understanding of other languages due to the multilingual nature of its training data.

However, its understanding of non-English languages is not as robust as its understanding of English. This is why it might be able to give a somewhat accurate prediction for Korean/Chinese text articles, but the accuracy is not as high as it would be for English text.

If you want to improve the accuracy of your model for non-English languages, you might want to consider using a multilingual BERT model, such as BERT-Base, Multilingual Cased (New, recommended). This model is trained on 104 languages and might give better results for non-English text.

Another option would be to use a language-specific BERT model, if one is available for the language you are interested in. For example, there are pre-trained BERT models available specifically for Chinese and Korean.

Lastly, you could also consider cleaning your dataset to remove non-English articles if your primary focus is on English news classification.

But to improve your score and if you wish keep your current data, then try to use the models like bert-base-multilingual-uncased or xlm-roberta-base.