# Building a Tic Tac Toe game which learns by itself

Let's assume we have a huge database gathered which contains winning sequences, which each player played and won. (Both the players are programmed, to obtain random numbers and attempt playing the game).

What would be the best way to use these winning sequences each player has gathered for future plays? Are we going to program one occurrence to play without using historical data and few other occurrences using historical data?

What would be the best way to save the winning sequences, the entire path which both the player One and Player Two has taken or only the sequence used by the winning player?

This is my first attempt to machine learning paradigm and hope some support could be taken!

• As you have quoted the question, it implies this is from a book or course. What is the subject matter? What are the subjects in the course that you have already been taught? You should expect to answer using that material. To me, it looks like Reinforcement Learning, but there are other options. Feb 25, 2017 at 9:24
• @NeilSlater This is not reinforcement learning, because there is a "huge database" already. The question is about how to use the data to learn, but not how to generate new data. Feb 25, 2017 at 11:00
• @StudentT: The "huge database" is perfectly good material to feed into a few types of RL algorithm - basic off-policy monte-carlo methods would work nicely here. RL is not just about learning from live experience. Feb 25, 2017 at 11:03
• For OP. The data look like typical outputs from Monte Carlo Tree Search. To me, it's like you've done MCTS and you're now using the data to update your model. Feb 25, 2017 at 11:04
• cs.stackexchange.com/q/70776/755
– D.W.
Feb 25, 2017 at 18:09

## 1 Answer

I took an introductory computer science course on Coursera a few years back where we had to use a Monte Carlo algorithm to build a tic-tac-toe AI. Instead of using historical data, we generated data using a function that would play through a series of games with random moves, but I think the principle is basically the same.

Each square on the board has a separate score associated with it. For each game completed, the scores change based on which player won. If the player designated as the machine player won, the machine player's score for each square on the board where it placed a mark would increase and the "other" player's score would decrease for every square where it placed a mark. Conversely, if the other player won, its score would increase for each square on the board where it placed a mark and the machine player's score would decrease. If the game is a tie, the number of points for each square would not change.

So, to answer your question, I think it is important to track both the winning and losing player's moves because we need to know not only what works but also what doesn't work.

Edit: I just remembered that my first implementation did not deduct points on the grid for games lost and because of this, my AI had an all-out "offensive" strategy. In other words, it focused on getting three-in-a-row without making any effort to block the human player. This is why it's important to track moves that result in a loss.