I am making a fun experiment in which a machine will mix different percentages of three juices – orange, apple and grape. After each mix is dispensed, a participant will taste the juice and rate it on a numeric scale, a score from 1 to 7.

Using the data collected, I would like to try and find the optimal juice mix programmatically.

For this I wish to implement a machine learning algorithm, that will both generate new ratios of juices to try, and using the response will try and find to the optimal juice mixture percentages.

What algorithm would you recommend me to use?


  1. I am planning on displaying this at an event, and estimate around 200 people to taste, at least.
  2. I am aware that due to the fact that different people have different taste preferences, I aim for the best tasting mixture that will fit most people.
  • 1
    $\begingroup$ Welcome to DataScience.SE! How are the participants rating; binary, percentage, rank? $\endgroup$
    – Emre
    Apr 17 '17 at 19:56
  • $\begingroup$ It's up to me. Right now I'm leaning towards a number scale, say 1-7. $\endgroup$
    – AlonMln
    Apr 17 '17 at 20:04
  • $\begingroup$ For bonus marks, consider the different cost of the three juices and maximise the expected profit :) $\endgroup$
    – Spacedman
    Apr 18 '17 at 7:26

As described, you have no data describing individual people (such as age, sex, shoe size), but are searching for an optimum value of the mix for the whole population. So what you want is a mix with the maximum expected rating, if you chose a random person to rate it from the population. In principle, this expected rating is a function taking two parameters e.g. $f(n_{apple}, n_{orange})$ - the amount of the third juice type is a not a free choice, so you only have two dimensions.

You can break down your problem into two distinct parts:

  • Taking samples from your population in order to find approximation to the function $f(n_{apple}, n_{orange})$

  • Using the approximation as it evolves to guide the search for an optimum value.

For a simple approach, you could ignore the second bullet point and just randomly sample different mixes throughout the event. Then train a regression ML on the ratings (any algorithm would do, although you'll probably want something nonlinear, otherwise you'll just predict one of the pure juices as favourite) - finally graph its predictions and find the maximum rating at the end. This would probably be fine when pitched as a fun experiment.

However, there is a more sophisticated approach that is well-studied and used to make decisions when you want to optimise an expected value of an action whilst exploring options - it is usually called multi-armed bandit. In your case, you would need variants of it that consider an "arm space" or parametric choice, as opposed to a finite number of choices that represent selecting between actions. This is important to you, since splitting your mix parameters up into e.g. in 5% steps, will give you too many options to explore given the number of samples you need to make. Instead, you will need to make an assumption that the expected rating function is relatively smooth - the expected rating for 35% Apple, 10% Orange, 55% Grape is correlated with the rating for 37% Apple, 9% Orange, 54% Grape . . . that seems at least reasonable to me, but you should make clear in any write-up that this is an assumption and/or find something published that supports it. If you make this assumption, you can then use a function approximator such as a neural network, a program like xgboost or maybe some Guassian kernels to predict expected rating from mix percentages.

In brief for a multi-armed bandit problem, you will use data collected as your experiment progresses to estimate the expected value for each choice, and on each step will make a new choice of mix. The choice itself will be guided by your current best approximation. However, you don't always sample the current top-rated value, you need to explore other mixes in order to refine your estimated function. You have choices here too - you could use $\epsilon$-greedy where e.g. 10% of the time you choose completely randomly to get other sample points. However, you might need something more sophisticated that explores more to start with and still converges quickly, such as Gibbs sampling.

One thing you don't say is at what level you are pitching this experiment. Studying the multi-armed bandit problem by yourself referring to blogs, tutorials and papers could be a bit too much work if this is for school science fair. If this all seems a bit too vague and a lot of work to study, then you can probably stick with a simple regression model from the data of a random experiment.

I suggest whichever approach you take, that you run some simulations of input data and see whether your approach works. Obviously there is a lot of guess work here. But the principle is:

  • Create a "true" model function - e.g. pick an imaginary favourite mix and make it score higher. Make it a simple and probably quite subtle function - e.g. score 5 for best result, and take away euclidean distance in "juice space" times a small factor (maybe 1.5) from it.

  • Create a noisy sampler that imitates someone in your experiment giving a rating to a specific mix. Ensure that the mean value from this matches the "true" function.

  • Try out your sampling and learning strategies, see how well they find the favourite mix.

I highly recommend this kind of dry run before putting your system to real use, otherwise you will have no confidence that your ML/approximator is working.

One more piece of advice about your estimator: You are expecting a large amount of variance in your data, and will not have a lot of samples. So to avoid over-fitting you will want to have a relatively simple ML model. For a neural network for example, you will probably want only one hidden layer with very few neurons in it (e.g. 4 or 5 might be enough). Finding a model sophisticated enough to predict a curve, but simple enough that it doesn't overfit given very noisy target outputs might take a few tries - this is the main reason why I suggest performing trial runs with simulated data.

  • $\begingroup$ +1. Neil and nice that you had mentioned that going through the multi-bandit problem itself for a school science fair project is bit too much :) $\endgroup$ Apr 17 '17 at 22:11
  • $\begingroup$ Wow thanks! I'm going to do some reading and I'll choose an algorithm. One question: How would you recommend choosing each time what mixture to rate each time? If I understood you correctly, you recommend choosing the highest valued predicted mixture which hasn't been tasted yet, while having a random chance to get a completely random mixture, to explore varied results in the beginning? By the way, this is an event where it is acceptable to invest a lot of time working on a project in advance, so I have time to work stuff out :) $\endgroup$
    – AlonMln
    Apr 18 '17 at 14:28
  • $\begingroup$ @AlonMin: If you decide to use a multi-armed bandit solver, then the strategy for choosing next mix will be part of that. $\epsilon$-greedy strategy is one such where you choose current best option most of the time ($p = 1 - \epsilon$), but otherwise choose completely randomly. A Gibbs sampling approach more generally favours values with better estimates, and can be adjusted to do so more or less extremely (lower "temperature" parameters are more sensitive to differences). $\endgroup$ Apr 18 '17 at 16:32

If you are interested, I recommend you try the Ryskamp Machine Learning (RLM) engine in your project. You can visit our site to learn more about it and, from there, you can get the source code (hosted on github) and give it a shot.

Note that the RLM is very different from traditional machine learning. Although some basic concepts still apply such as inputs, outputs, etc. but one (out of 7 breakthroughs) of what really sets it apart is the core algorithm. The RLM does not apply the same mathematical algorithms used by traditional ML rather it goes about the problem in a logical approach. See the 7 breakthroughs on our site to learn more about the differences.

As for your project, here are some suggestions on how you can apply the RLM:

  1. You could have it do Supervised Training. Like you will do in traditional ML, you should gather the participants data together with the ratings you mentioned. With the dataset in hand, you can then proceed training the RLM with it. You could set the input to always 1 while the output to be the three fruit ratios. The reason you have 1 as your input is because you are trying to find the best ratio for the entire population. Where you to find the best ratio for certain groups of participants, you could then have different values on your inputs (i.e., 1 = Participants Age >= 30, 2 = Participants Age < 30, 3 = etc..). That way you can find what fruit ratio was optimal for group 1, group 2 or more.
  2. Or, you could do Unsupervised Training. Instead of preparing a predefined list of ratios to let your participants try and rate, the RLM can do that as it will try different ratios until it comes up with the optimal one. You can still follow the same inputs and outputs I mentioned in #1. Although, I worry that having only 200 participants might not be enough to fully find the optimal fruit ratio using this method.

On both options, the participant's rating (1-7 as you mentioned) is the Score you will provide to the RLM.

I hope this helps. If ever you need any technical assistance or feedback, please feel free to reach out to me.

P.S.: Although the RLM has been recently open sourced, there are some limitations to its use. Please be sure to read the license.

  • $\begingroup$ I'm not sure if this answers the OP's question, as they asked for an algorithm, and this appears to be more of a pre-packaged product. However, out of interest have you tried your RLM against some trickier problems - such as those in universe.openai.com ? It looks like it behaves as a basic Reinforcement Learning solver, so might be able to do some of those tasks, and you would have a better idea of its relative performance (I am skeptical of the claims on your product site, because the example challenges RLM solves seem trivial compared to state of the art) $\endgroup$ Apr 19 '17 at 9:34
  • $\begingroup$ Hi Neil, Yes, the RLM is a complete machine learning system. No, it is not a product. It is just a machine learning system. We are a business and have limited resources to spend on various proofs. We have solved much trickier problems in real world scenarios. I will suggest to my boss that we look at linkuniverse.openai.com. The examples on the site are meant to be simple to understand. However, I would challenge you to find another machine learning engine that can solve Lunar Lander and a Maze with the same algorithm. The examples... $\endgroup$
    – Randolph
    Apr 20 '17 at 5:57
  • $\begingroup$ ...are meant to show the RLM’s general learning capabilities. It has similarities to a neural network and a reinforcement learner yet it is completely unique as a system and the results speak for themselves. If you have time and interest in proving it out we give out prizes for developers that build cool new tests of our engine versus their favorite engine. We will publish the results and give the prize even if you can beat our engine with something else. Talk to me offline if you are interested. $\endgroup$
    – Randolph
    Apr 20 '17 at 5:57
  • $\begingroup$ OP – if you want to write your own algorithm, the RLM would not be right for you. If you want to get machine learning working up and quickly it is a great route and I am happy to support you. $\endgroup$
    – Randolph
    Apr 20 '17 at 5:57
  • $\begingroup$ Hmm, it's C# - do you know if it compiles and runs under Mono on a Mac? $\endgroup$ Apr 20 '17 at 7:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.