2
$\begingroup$

Consider the following problem, simplified:

Let's say that you have some data where you have only 5 columns/attributes.
One of them can be seen as a reward. How well we performed at a given run.
The other two attributes, let's say x and y are our input to the system. A human hand picked those x, y values and they were recorded.
Another two attributes are observations from sensors. These are not inputs and we have no control over them. Let's call them ob1 and ob2.
All are real values (nothing is discrete).

One idea is to look at this dataset as an ordered dataset.
We are asked to choose input values x and y that maximize our reward.

We have available ~70.000 instances of this five-tupled dataset

This is one approach that comes into mind but not sure if it is correct in principle or the simpler one.
We could build a predictor which takes as input the four attributes (x, y, ob1, ob2) and has as target the reward.
Then try a repetitive process for the inputs by using reinfocement learning (?) to get the inputs that maximize the reward?..

$\endgroup$
3
  • $\begingroup$ Look up "reinforcement learning" and "active learning". $\endgroup$
    – Emre
    Feb 17, 2017 at 22:52
  • $\begingroup$ Are x and y continuous? If not, how many choices do x and y have? $\endgroup$
    – Icyblade
    Feb 18, 2017 at 1:44
  • $\begingroup$ yes continuous, not discrete $\endgroup$ Feb 18, 2017 at 1:44

1 Answer 1

1
$\begingroup$

It seems like a reinforcement learning question with continuous action space. (ob1, ob2) is the observation, (x, y) the action, and reward the reward.

You may refer to this paper from DeepMind, which provides a general deep Q-learning way solving this type of question, including some physics tasks.

However, as I don't have your data, I'm not sure if you have sufficient instances. ~70,000 instances with 4 features is (roughly) sufficient for general ML problems, but in reinforcement learning world, people use a simulation environment which is able to generate infinite samples, thus ~70,000 instances may be the bottleneck.

If you encounter such problem, you may try classic reinforcement learning (without "deep") techniques.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.