I am not an expert on Machine Learning, Neural Networks or NEAT. In fact, I probably have no clue what I'm talking about. My question is if you can make a learning AI that learns to play complex multiplayer games and possibly outpreform humans. If it is possible, could you also recommend a language or languages to make this AI in? (I know I'll probably have to take a VACation for botting, but it's something I feel like we should try.)
The answer is yes:
Example of a neural network outplaying human players in DOTA. I haven't been able to find much regarding what kind of neural network but here is what is on the OpenAI website. If you're a beginner you can learn some basic architecture and design principles in neural networks using Python and Keras (a neural network library).
The question isn't whether this can be done. The question is how would you train such a model?
If you read the whitepapers on things like Alpha Go and the like, those models are just reinforcement learning run on massive hardware. Since Alpha Go was backed by Google, it was nothing for them to build a simple model and then tell it to go and play 10,000,000,000 games (or whatever) since it could do that in a very short amount of time (and many more games than any one human could possibly hope to achieve). Plus the straightforward nature of Go makes it easy to write software the simulates a game.
So, you would have to think about how your model (any model) could replicate this type of learning in an automated way. Are there mathematical representations of CS:GO that you can use to feed into the model? Do you have a way for your model to play a ton of these games automatically so that it can learn from it's own actions? You would have to answer both of these questions in a meaningful way in order to verify that there is a path forward with this effort.