Is it possible to use NEAT networks for solving video games?

Question

Sorry to start such an unspecific question, but I am slightly lost in the big topic.

My tutor proposed to chose neural networks for my final project, and we started by building a CNN for detecting hand written digits in python.

My own idea was to build another neural network that can play video games of ascending difficulty, for example pong -> snake -> megaman [with tools for extracting in game data] or driving in GTA San Andreas similar to what Sentdex did. (For GTA, I would try something different to Sentdex as I would train the neural network by reinforcement learning in artificial conditions, starting with staying in a straight lane and then moving on to curves, traffic etc.) I made my own version of snake, which I could use for additional input and seems like a realistic minimal goal.

But now, I am not so sure which approach I should take for this. I took a look at Sethblings MarI/O and read the NEAT paper. Then, I read the book "Machine Learning in Python" explaining a lot of concepts, and realised that neural networks and especially GA's are only a small part of possible algorithms. Also, I am not sure how much Sethblings network actually thinks and how much it just randomly adapted to the level.

Do you think that the following idea is realistic?: "I build my own neural network in python based on NEAT, then test it on video games / tasks of increasing difficulty and try to tune parameters for performance"

I could also imagine comparing NEAT to an implementation of Alexnet for the same tasks. The whole goal is not to create the perfect NEAT but instead just make one and see how far it takes me. I have a lot of time and motivation at hand, but I don't want to exceed the boundaries of a high school final and my own possibilities.

Feedback welcome and thank you in advance

Answer 1

Implementing the NEAT algorithm for playing video games is quite realistic, especially considering that it is not novel, so there are extensive resources and tutorials to help you. In fact, there are publications that will walk you through exactly what you want to do, such as this paper on using the NEAT algorithm for playing Pong.

Of course designing a self-driving car would be significantly more advanced than playing the 2D games you mentioned, but it appears that much of the work for developing AI in GTA V exists.

The person you mentioned, Sentdex, has a github page for developing GTA V code. However, note that:

We read frames directly from the desktop, rather than working with the game's code itself. This means it works with more games than just GTA V, and it will basically learn (well, attempt to learn...) whatever you put in front of it based on the frames as input and key presses as output.

This is a very primitive approach.

I am by no means an expert on self-driving cars, but after some Googling, deep drive appears to be the premier simulated self-driving car development platform. If you are going to develop the GTA network, I suggest checking out it out.

---

As for comparing to an implementation of AlexNet:

Again, I am not an expert on self-driving cars, but I would expect an ensemble of networks to be used for this type of task, so I imagine an object detection network or semantic segmentation network to be used in conjunction with a network designed with the NEAT algorithm.

For instance, you can use a fully convolutional AlexNet model to detect objects such as pedestrians/cars/sidewalks/etc and feed that data into another network that also takes other inputs such as your current speed, wheel angle, road conditions etc that combines all this information and translates it into actions such as applying the brakes, turning etc.

AlexNet is a Convolutional Neural Network (CNN), which is primarily used in the field of vision processing. The NEAT algorithm typically generates Multilayer Perceptrons (MLP), which are not used individually for machine vision challenges; they are different tools for different tasks. It would be like asking, "which is better: a hammer or a screwdriver". The answer depends on the task.

---

TL;DR:

Yes, using NEAT for video game AI development is a good introductory project for some of the more advanced topics in machine learning and deep learning.