2
$\begingroup$

Consider the following problem, simplified:

Let's say that you have some data where you have only 5 columns/attributes.
One of them can be seen as a reward. How well we performed at a given run.
The other two attributes, let's say x and y are our input to the system. A human hand picked those x, y values and they were recorded.
Another two attributes are observations from sensors. These are not inputs and we have no control over them. Let's call them ob1 and ob2.
All are real values (nothing is discrete).

One idea is to look at this dataset as an ordered dataset.
We are asked to choose input values x and y that maximize our reward.

We have available ~70.000 instances of this five-tupled dataset

This is one approach that comes into mind but not sure if it is correct in principle or the simpler one.
We could build a predictor which takes as input the four attributes (x, y, ob1, ob2) and has as target the reward.
Then try a repetitive process for the inputs by using reinfocement learning (?) to get the inputs that maximize the reward?..

$\endgroup$
  • $\begingroup$ Look up "reinforcement learning" and "active learning". $\endgroup$ – Emre Feb 17 '17 at 22:52
  • $\begingroup$ Are x and y continuous? If not, how many choices do x and y have? $\endgroup$ – Icyblade Feb 18 '17 at 1:44
  • $\begingroup$ yes continuous, not discrete $\endgroup$ – George Pligoropoulos Feb 18 '17 at 1:44
1
$\begingroup$

It seems like a reinforcement learning question with continuous action space. (ob1, ob2) is the observation, (x, y) the action, and reward the reward.

You may refer to this paper from DeepMind, which provides a general deep Q-learning way solving this type of question, including some physics tasks.

However, as I don't have your data, I'm not sure if you have sufficient instances. ~70,000 instances with 4 features is (roughly) sufficient for general ML problems, but in reinforcement learning world, people use a simulation environment which is able to generate infinite samples, thus ~70,000 instances may be the bottleneck.

If you encounter such problem, you may try classic reinforcement learning (without "deep") techniques.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.