2
$\begingroup$

I'm studying a paper about Named Entity Recognition. The following is a part of the abstract:

To assess the robustness of NER systems, we propose an evaluation method that focuses on subsets of tokens that represent specific sources of errors: unknown words and label shift or ambiguity.

I don't know what "label shift" definition is. The paper doesn't explain it and I can not find anything that I could understand by Googling it.

$\endgroup$
2
$\begingroup$

Label shift is the opposite of a covariate shift.
In this case, the assumption is that even though the feature distribution remains the same, the Label distribution might changes.
e.g. Symptoms --> Diseases

It can be different for different country (based on medical education of the Country/Doctor)
It can change with time also based on advancement in Medical knowledge

Similar logic can be built for "Words --> Slang". It can change with time due to the acceptance of these words.

Read the references for a formal explanation.

References-
Dive into Deep Learning
Detecting and Correcting for Label Shift with Black Box Predictors

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.