I have a doubt solving a test. The idea here is to demonstrate the NLP and Machine Translation abilities.
The Dataset is a multilingual, multi-context set of documents. The dataset is divided on context categories (Wikipedia, conference_papers, Amazon Reviews, etc.,) and on languages.
The objective is to create a document cartegorization classifier (in Python) for the different contexts of the documents. The classifier has to be done at context level, regardless of the language the documents are written in.
An important fact is that The dataset original has been modified and a document Never is repeated in 2 languages.
I have 2 ideas on mind to solve that:
- Train on all the data creating a multilingual classifier
- Doing language detection first and use monolingual models later.
What could be a reasonable approach to doing text classification for multiple languages?