1
$\begingroup$

In NLP, languages are often referred as low resource or high resource.

What these terms mean ?

$\endgroup$
3
$\begingroup$

High resource languages are languages for which many data resources exist, making possible the development of machine-learning based systems for these languages. English is by far the most well resourced language. West-Europe languages are quite well covered, as well as Japanese and Chinese. Naturally low-resource languages are the opposite, that is languages with none or very few resources available. Unsurprisingly many languages from Africa and other poorest parts of the world are low-resources. But this is also the case for some extinct or near-extinct languages and many local dialects. There are actually many languages which are mostly oral, for which very few written resources exist (let alone resources in electronic format); for some there are written documents but not even something as basic as a dictionary.

There are many different types of resources which are needed in order to train good language-based systems:

  • a high amount of raw text from various genres (type of documents), e.g. books, scientific papers, emails, social media content, etc.
  • lexical, syntactic and semantic resources such as dictionaries, dependency tree corpora, semantic databases (e.g. WordNet), etc.
  • task-specific resources such as parallel corpora for machine translation, various kinds of annotated text (e.g. with part-of-speech tags, named entities, etc.)

Many types of language resources are costly to produce, this is why the economic inequalities between countries/languages are reflected in the amount (or absence) of language resources. The Universal Dependencies project is an interesting effort to fill this gap.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.