Training an AI to play Starcraft 2 with superhuman level of performance?

Question

I'm interested in working on challenging AI problems, and after reading this article (https://deepmind.com/blog/deepmind-and-blizzard-open-starcraft-ii-ai-research-environment/) by DeepMind and Blizzard, I think that developing a robust AI capable of learning to play Starcraft 2 with superhuman level of performance (without prior knowledge or human hard-coded heuristics) would imply a huge breakthrough in AI research.

Sure I know this is an extremely challenging problem, and by no means I pretend to be the one solving it, but I think it's a challenge worth taking on nonetheless because the complexity of the decision making required is much closer to the real world and so this forces you to come up with much more robust, generalizable AI algorithms that could potentially be applied to other domains.

For instance, an AI that plays Starcraft 2 would have to be able to watch the screen, identify objects, positions, identify units moving and their trajectories, update its current knowledge of the world, make predictions, make decisions, have short term and long term goals, listen to sounds (because the game includes sounds), understand natural language (to read and understand text descriptions appearing in the screen as well), it should probably be endowed also with some sort of attention mechanism to be able to pay attention to certain regions of interest of the screen, etc. So it becomes obvious that at least one would need to know about Computer Vision, Object Recognition, Knowledge Bases, Short Term / Long Term Planning, Audio Recognition, Natural Language Processing, Visual Attention Models, etc. And obviously it would not be enough to just study each area independently, it would also be necessary to come up with ways to integrate everything into a single system.

So, does anybody know good resources with content relevant to this problem? I would appreciate any suggestions of papers, books, blogs, whatever useful resource out there (ideally state-of-the-art) which would be helpful for somebody interested in this problem.

Thanks in advance.

Answer 1

You pose an interesting question. The problem that I see is that even if you develop all the items you mentioned (and I'm not sure you need all of them) you wouldn't have the computing power needed for this.

You see, most of these items are based on reinforcement learning. Which means that the models are given a relatively small set of data (roughly speaking, the rules of the game) and then are set off to play millions & millions of games. Read the whitepaper that was written by Google on beating Go. They basically admit that there whole point was to write something very, very simple that could learn on it's own and then just set it free on Google's massive power. It's nothing for them to say, "let's have this robot play 10 million games". So the level of sophistication from your initial model is pretty low, you just need to make it an exceptional learner.

So you make an exceptional learner of Starcraft - then what? How are you going to have the power to make your algorithm to play millions of games?

Answer 2

This answer is based on my opinion and others', it might not be the answer you expect.

The problem you want to solve is probably in the field of Artificial General Intelligence. The problems require AGI to solve are informally known as AI-complete.

AI-complete problems are hypothesised to include computer vision, natural language understanding, and dealing with unexpected circumstances while solving any real world problem.

Currently, we're not ready to solve these problems, and if Gartner is right, the hype won't come to peak in 10 years.

This is the Gartner hype curve in 2017, and AGI is still in its early phase.

The problem is, at the moment, our state-of-the-art AI models (Deep Learning) don't have the ability to explain their decision or reason about their behavior, which makes learning highly inefficient.

In addition, the Reinforcement Learning framework is way too simple and it ignores the complexity of human environment. You might find this answer useful.

Geoffrey Hinton, one of the leaders of the field, said in an interview that AI needed to start over.

"My view is throw it all away and start again,"

"I don't think it's how the brain works. We clearly don't need all the labeled data."

That's it. If one day AI can learn how to play Starcraft 2, it probably learns in a very different way from what it learns today.