I’m the lead author of that paper :)
Mislabeled data can detrimentally effect the performance of ML models. The primary takeaway of that paper is that you need to use accurately labeled test sets when benchmarking. You also should train your models on accurately labeled data.
The easiest (one-line-of-code) way to find mislabeled data is via the cleanlab data-centric AI python package: https://github.com/cleanlab/cleanlab which works for any dataset you can train a classifier on for most data formats, ML and deep learning frameworks, and data modalities, e.g. image, text, tabular, and audio data.
Find label issues in 1 line of code
from cleanlab.classification import CleanLearning
from cleanlab.filter import find_label_issues
# Option 1 - works with sklearn-compatible models - just input the data and labels ツ
label_issues_info = CleanLearning(clf=sklearn_compatible_model).find_label_issues(data, labels)
# Option 2 - works with ANY ML model - just input the model's predicted probabilities
ordered_label_issues = find_label_issues(
labels=labels,
pred_probs=pred_probs, # out-of-sample predicted probabilities from any model
return_indices_ranked_by='self_confidence',
)
Train a model as if the dataset did not have errors -- 3 lines of code
from sklearn.linear_model import LogisticRegression
from cleanlab.classification import CleanLearning
cl = CleanLearning(clf=LogisticRegression()) # any sklearn-compatible classifier
cl.fit(train_data, labels)
# Estimate the predictions you would have gotten if you trained without mislabeled data.
predictions = cl.predict(test_data)
Documentation and runnable tutorials for cleanlab: https://docs.cleanlab.ai/
The above approaches find issues. To correct label issues in your dataset...
The above approaches, just find issues and train without them. To correct the labels or train a model on a corrected dataset (that still includes all the errors, but now corrected with the right label), there is a no-code tool for that called Cleanlab Studio (https://cleanlab.ai/studio) for which I am also an author.