0
$\begingroup$

I am tackling a RL problem (relaxed version of Space Fortress) with DQN. The usual approach would be to feed pixels into a CNN but that is usually very slow. I am considering feeding symbolic features such as coordinates, velocity and angles of the relevant elements into a Multi Layer Perceptron because then the training would be faster and I could try more things quicker. My idea is that the perceptual part of the task is not that interesting to learn here and I would like to focus on the RL part instead (I am experimenting with Hierarchical RL).

Given that with all the symbolic features that I would provide it would possible to reconstruct the full image and hence the state of the environment, would there be any reason that a CNN pixel based approach would be better?

PT: The environment has a fixed number of elements

Thanks

$\endgroup$

1 Answer 1

1
$\begingroup$

If the features are able to fully define the image, there would be no reason to use a pixel-based approach. You are essentially performing lossless compression and (nearly) instantly acquiring a feature set, so you would see a dramatic performance difference between the CNN and MLP.

If you needed a quick solution, CNNs are great out-of-the-box algorithms, but if you are willing to put in the time to develop an MLP, it would provide a performance boost without losing accuracy.

$\endgroup$
4
  • $\begingroup$ Actually I can't fully define the image because I don't provide some constants. For example, there is a element of fixed size that moves around so I only provide coordinates and speed but not its size. I considered that it is useless to always feed the same value to an input neuron. Is this consideration correct? $\endgroup$ Commented May 6, 2018 at 7:29
  • 1
    $\begingroup$ That is correct. However, if you needed to encode size data, you can pass the coordinates of the object's bounding box rather than its centroid. $\endgroup$
    – Ben
    Commented May 6, 2018 at 11:28
  • $\begingroup$ But why would I need to encode size data? If the moving square is always the same size then feeding only e.g. its upper left corner coordinates should be more than enough, right? $\endgroup$ Commented May 6, 2018 at 11:45
  • 1
    $\begingroup$ It depends on your task and environment, but if your object is rotating then the bounding box coordinates could be important. Another situation where you would use multiple coordinate points is any time it would eliminate ambiguity. For instance, let's say you have 10 objects total: 6 are small and 4 are large. Now let's say one of the 6 small objects turns into a large object, then simply feeding a point would not be sufficient. Other than situations like those, a single point is probably adequate. $\endgroup$
    – Ben
    Commented May 8, 2018 at 12:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.