0
$\begingroup$

Background: In collecting a dataset of a specific unit ordered by a numeric variable, it is possible that the upper 'cloud' of the dataset is correct, while the 'tail' seems inaccurate.

I can thus trust the upper bounds of the dataset, while I deem the mid- and lower ranges to be rather questionable.

Question: Is there a data scientific term for this phenomenon? If so, how is it called?

Example: I use various sources to gather a list of major publishers of scholarly journals. In total, I found 150 publishers that publish at least 50 journals.

At the top, I found the publisher AAA with 3.000 journals, BBB with 2.500 journals, CCC with 1.900 journals, DDD with 1.500 journals etc.

With my industry knowledge, I can confirm that the top 20 is likely to be accurate.

However, in the lower ranges, there are publishers like XXX with 51 journals, YYY with 50 journals, ZZZ with 50 journals etc. Many of them are rather obscure, and even as an expert I may have never heard of them. I can imagine that the 'tail' is rather inaccurate with large-scale omissions, such as publishers from the Global South.

I thus tend to trust the top 20 or so of that list, but not the ones that rank between #21 and #150.

Improve this question
$\endgroup$
1
  • $\begingroup$ Please notice that class imbalance has absolutely nothing to do with the "accuracy in the tail" phenomenon you seem to describe here. Rare cases in imbalanced settings are not considered inaccurate - on the contrary, very often they are the actual cases of interest (patients, fraud cases, faulty machines etc) $\endgroup$
    – desertnaut
    Feb 10, 2021 at 13:46

1 Answer 1

Reset to default
1
$\begingroup$

The case described can be cast as imbalanced dataset problem, or rare events problem.

Even more generally as highly non-uniform underlying distribution problem (which is an umbrella term for both cases).

References:

  1. Machine Learning Tips: Handling Imbalanced Datasets
  2. Handling imbalanced datasets in machine learning
Improve this answer
$\endgroup$
5
  • $\begingroup$ Class imbalance has absolutely nothing to do with the "accuracy in the tail" phenomenon OP seems to describe. Rare cases in imbalanced settings are not considered inaccurate - on the contrary, very often they are the actual cases of interest (patients, fraud cases, faulty machines etc) $\endgroup$
    – desertnaut
    Feb 10, 2021 at 13:48
  • $\begingroup$ @desertnaut IMO OP describes a class-imbalanced problem, in the sense that certain labels have very few samples relative to other labels. $\endgroup$
    – Nikos M.
    Feb 10, 2021 at 16:11
  • $\begingroup$ OP states that "the 'tail' is rather inaccurate with large-scale omissions" and "I thus tend to trust the top 20 or so of that list, but not the ones that rank between #21 and #150". Such characteristics do not belong to a typical class imbalance setting; there are no trust issues in class imbalance. $\endgroup$
    – desertnaut
    Feb 10, 2021 at 16:16
  • $\begingroup$ This is not to say that there is not class imbalance here - indeed there is; but it does not capture the specific characteristics of data trust/reliability as described by the OP. $\endgroup$
    – desertnaut
    Feb 10, 2021 at 16:19
  • $\begingroup$ @desertnaut The "trust" issue I attribute to the imbalance (per OP's emphasis), but feel free to submit another answer $\endgroup$
    – Nikos M.
    Feb 10, 2021 at 16:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.