2
$\begingroup$

Recently I got interested in the process of data cleansing and specifically in record linkage.

Thus far I read about deterministic and probabilistic approaches to deduplicate data sets and to some lesser degree also about machine learning methods. It struck me that the key part of all algorithm basically introduce a metric space. Through the metric space every two data points can be assigned a distance. The distance is then basically a measure of how close these two data points are related to another.

However I do wonder, if there were not also different kinds of algorithms that do not work on this principle?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ I suppose you could use a decision tree for argument's sake. Metric spaces are good; use them. $\endgroup$
    – Emre
    Commented Nov 3, 2017 at 6:21
  • $\begingroup$ @Emre, I don't want to use, but understand them :P, there are also many other things I would like to understand better. But thanks for you suggestion on decision trees, I will read into them. $\endgroup$
    – Imago
    Commented Nov 3, 2017 at 9:27

1 Answer 1

Reset to default
1
$\begingroup$

One option is fingerprinting. If two objects have the same fingerprint, they are probably the same object. Depending the technique used, the fingerprint can not tell about approximate duplicates.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.