In RL context, I know that features are explanatory variables that represent or describe the states. If I want to do feature engineering on atari games and use it to solve RL task, how should I implement it? Can anyone point me out any starting code to do this?

for instance, in cartPole, features can be cart position, cart velocity, pole angle, pole velocity at tip. we know the observation from environment, but how should I represent those features in code?

In my experiment, I am building agent with policy gradient algorithm to solve RL task, I want to know how should I represent or define hand-crafted features as vector numerically in code. Can anyone show me possible starting code for representing features in pong game? Any thoughts?

  • $\begingroup$ Do you mean that you are building an agent that will play pong? And are you looking to encode the game as text to input into that agent. Or some similar principle? $\endgroup$
    – Bruce
    Jan 12 at 6:34
  • $\begingroup$ @Bruce yes, I built agent to play pong games. How should I extract features that represent cart position, cart velocity, pole angle, pole velocity from input image (game screenshot)? Any idea? $\endgroup$
    – Jerry07
    Jan 18 at 4:29
  • $\begingroup$ @Bruce I am also interested in this, do you know get features ready for training agent to play the games? Any thoughts? $\endgroup$
    – Hamilton
    2 days ago
  • $\begingroup$ @Hamilton and Jerry - I am not entirely clear what the core focus of the question is, and I wonder that the two of you might be after different issues. But, I have added and answer that should either be useful, or, if it is not useful, it should clarify what it is that you are wanting to ask about. $\endgroup$
    – Bruce
    2 days ago

As I understand the question - a lot of this is about simple vision processing.

If you already had a access to the position of the ball and the position of the two paddles, one has almost everything required to be able to play the game - the rest is machine learning.

In the simplest sense the agent gets a sequence of tuples of 4 numbers, two for the ball and one each for the paddles. And it has to decide whether to move its paddle up or down. Assuming we are not trying to get the agent to double guess its opponent, this decision could be made in an entirely stateless manner. One just needs to decide up/down given the four numbers.

I assume we are talking about classic pong?

How to image process this?

I am assuming you are not intending for the agent to learn the vision processing? That would be a different discussion.

This problem is very simple in terms of modern vision processing.

  1. each paddles is a solid block that is the only thing inside a rectangle on the screen.

  2. the ball in play is a simple square inside an otherwise empty region of the screen. (Mild problem if as it crosses the mid line).

There may well be faster ways to do it - but the simplest way is to scan a line along the middle of each vertical rectangle looking for a section that has a high average intensity.

Similarly for the ball - there are different search patterns, but one could scan down a collection of vertical lines looking for a similar section of high average intensity.

What I am describing above is essentially a simplified version of finding a shape on the screen by using convolution/correlation between the shape you want and the screen.

If you want to read the block-characters on the screen, like the score, then a similar approach will make it possible to tell the difference between each of the characters. Worst case - you know exactly where they are to just look for correlation between a template of each character and the image on the screen.

I am assuming that you are working from a clean bit-mapped screen shot, rather than, for example, a camera pointed at the screen of the game.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.