0
$\begingroup$

Currently I am working on improving a stage on a data processing pipeline. The source data has a large number of fields and is getting normalized into a simpler entity. This entails that in many cases a destination field value may be copied from arbitrary input fields, according to the context.

My idea was to regress a binary output sources-destinations matrix that associates the possible source fields to the possible source destinations.

I was wondering: is this a problem that has been tackled before? Is there anything in scientific literature that is worth noting?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

What you are describing could be modeled as bipartite graph, one set of nodes connects to another set of nodes. Thus, it becomes a bipartite graph matching problem.

As the size of each graph grows it quickly becomes intractable, then approximate nearest neighbor search might be useful.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.