I have "millions" of items each having N binary features. When a feature is "0" it could be that the information is simply missing. So, given the data with the currently observed 1's, I would like to have a probability of the "0" features being "1".

I am thinking this can be a Neural network with all features as input and same as output. But then I don't know how the training would work. I don't have ground truth.

I would like some help expressing my problem and hopefully not reinvent the wheel. Is this is a classical problem in ML, and what approach can be applied?

  • $\begingroup$ What about the ground truth? I mean it is not clear to me whether you have a dataset correctly labeled to make a model learn against. $\endgroup$ – tagoma Dec 14 '17 at 21:31
  • $\begingroup$ We don't have a ground truth. $\endgroup$ – DED Dec 15 '17 at 14:14

A simple approach could be the following: suppose $i \in \{0,1\}^d$ is the vector you want to predict which of the $0$ entries could be $1$ and $j \in J$ the rest of the feature vectors. Take the $k$ nearest neighbors, under some suitable distance (Jaccard, Hamming, Manhattan distance). For each $0$ entry the probabilities could be the percentage of the $k$ nearest neighbors that have $1$ in the corresponding entry.

This problem has been extensively study in the collaborative filtering community. The best known example being the Netflix Prize. This blog post provides a nice explanation of this approarch for binary data.

Another, more involved, approach is matrix completion, in particular check this reference. If you are into deep learning check this.

  • $\begingroup$ Brilliant! this seems to be exactly my setting. I will read through your pointers, thank you. $\endgroup$ – DED Dec 15 '17 at 16:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.