0
$\begingroup$

I've two different datasets A(x1, x2, x3...xn) and B(y1, y2, y3...yn). Each instance in A is linked with an instance in B i.e. only one unique pair exists. 

x1 -> y1
x1 -> y2
.     .
x2 -> y3
x3 -> y3
x4 -> y4
x5 -> y5
predict(xnew  -> ynew)?

Also, instances can be duplicate in A as well as B. If I get a new instance in A and a new instance in B, I want to predict whether they both will be linked or not. Please suggest a way to do this. Thanks in advance.

| improve this question | |
$\endgroup$
  • $\begingroup$ If I understood correctly there cannot be any solution to your problem, since, each pair is unique without any additional attribute information. There are link prediction algorithms but they reside on the fact of common links. In your case, every 2 nodes are linked (e.x. a1 -> b2) without any other possible linkage, leaving no room for link inference between the new nodes and the already nodes since there are no connections. Each A node is on it's own and B as well. If that's not the case please provide more information to your question. $\endgroup$ – Grzegorz Oct 10 '19 at 17:00
  • $\begingroup$ @Grzegorz pls look at the edit $\endgroup$ – daemonkiller May 15 at 16:33
0
$\begingroup$

Make X as feature
Y as Label
Build a model on it with 80% example.
Predict Y for 20% and check if it is very close to Y_actual.
If you are getting a good score. Then you are good.

Predict Y for new X
If Y_pred is close to Y_actual. You may assume both are linked.

| improve this answer | |
$\endgroup$
  • $\begingroup$ tried this already, does not seem to be good approach for real valued numeric data. $\endgroup$ – daemonkiller May 17 at 13:52
0
$\begingroup$

From the top of my head I think you could build something similar to an autoencoder.

$X$ as input, $Y$ as label: $Y' = f_\theta(X) \approx Y$ as label, $f_\theta$ is a Neural Network, $\theta$ the weights and your loss $\mathcal{L} = d(Y,Y')$, where $d$ is some convex distance measure like Mean-Squared-Error or Binary-Crossentropy (if you scale your output and labels between $(0,1)$).

If you have new data $\{\hat{X},\hat{Y} \}$, feed $\hat{X}$ to your network $f_\theta$ and look for the sample in $\hat{Y}$ which has the minimal distance $d(\hat{Y},\hat{Y'})$

| improve this answer | |
$\endgroup$
  • $\begingroup$ How is this different from training conventional ML models like SVM, Decision trees? Because this approach is failing for me. $\endgroup$ – daemonkiller May 17 at 19:03
0
$\begingroup$

This seems to me now to be a typical case of a bipartite graph. In the real world, this can be viewed with customers and products; where each customer is linked to a product that they bought, but node customers or product nodes cannot be linked to each other.

I do not know the SOTA for that, but a useful tool to check with Python is NetworkX. Perhaps, you could create a bipartite graph from your data and use one of their link prediction algorithms.

| improve this answer | |
$\endgroup$
  • $\begingroup$ seems like a promising solution! Will try it definitely. Thanks! $\endgroup$ – daemonkiller May 17 at 19:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.