Being still a bit new to neural networks, I wish to use some form of machine learning but I am not sure which one is most suited.

I have a toy-example of what I am working on. Tips are much appreciated.

Assuming there are chunks of data (not pixels of an image :)) which all are based on a root variable, being 'a'. This 'a' is the 'thing' and the other variables are aspects.

Also assuming it's a finite set of things and that each thing 'a' has four ('b', 'c' and 'd') aspects.

I wish to make a program that learns that thing ('a':1) often occurs with ('b':0.2), ('c' :0.8), and ('d':0.3). There is a catch however, I want there to be a certain conditional structure underlying the co-occurrence of a and the respective values of (b, c, d).

For instance:

('a':1) often     (prob. 0.7) occurs with ('b':0.2), ('c':0.8), and ('d':0.3)
('a':1) sometimes (prob. 0.6) occurs with ('b':0.2), ('c':0.5), and ('d':0.7)
('a':1) sometimes (prob. 0.4) occurs with ('b':0.3), ('c':0.4), and ('d':0.2)
('a':1) rarely    (prob. 0.2) occurs with ('b':0.5), ('c':0.3), and ('d':0.8)

For a given ('a':2) this would all be different. There is no implied necessity for the values of b, c and d to sum to 1. They are mere placeholder values representing aspects (0.2 could be green, 0.8 purple, etc.).

I want to make sure the program understands that there is an inherent dependency between b, c and d, as well as an 'a':x given (b, c, d).

If anyone has suggestions based on this, I would be very grateful. Thanks!

EDIT: I am working in python and was thinking about dictionaries as the data-structure, but if there are better solutions (also depending on the algorithm perhaps) I am all ears.

  • $\begingroup$ So your numbers are not number and represent more levels of an factor. You may transform you data to four factors with levels a1, a2, a3,.. b1,b2,... etc. Then ist seams you are looking for association rules. Check e.g. apriori algorithm. $\endgroup$ – Marmite Bomber Feb 8 '18 at 17:10
  • $\begingroup$ @MarmiteBomber Thank you for your comment! I will think about it, I have to see how it maps to my problem, I have somewhat oversimplified it for this questioning. Perhaps what you propose is suitable, otherwise I will edit the qeustion. Thanks regardless! $\endgroup$ – nick88 Feb 8 '18 at 21:43

To me, this describes conditional probabilities. And the first thing that springs to my mind when it comes to learning conditional probabilities is a Bayesian network (BN), which is an example of a probabilistic graphical model. In fact, BNs can be fitted using Maximum Likelihood Estimation (MLE), which corresponds to counting occurrences (similar to "Naive Bayes", if you have used that). An excellent book on this is Barber.

Another model that corresponds to your probabilistic problem, and which is more neural network-like, is a Restricted Boltzmann Machine (RBM):

enter image description here


Once again, this involves learning conditional probabilities, like $p(v_0 | h_0, h_1, h2), p(v_1 | h_0, h_1, h2), p(v_2 | h_0, h_1, h2), \ldots$. This is a nice introduction to them. Unlike neural networks, which are trained using backpropagation and are not probabilistic, RBMs are indeed probabilistic and are trained using Contrastive Divergence.

  • $\begingroup$ Thanks @A. G. this looks interesting, I will try it out and post any results (if they're worth sharing :)). $\endgroup$ – nick88 Feb 14 '18 at 14:03
  • $\begingroup$ I'd be interested to hear how it goes! $\endgroup$ – A. G. Feb 14 '18 at 17:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.