I'm trying to learn machine learning with tensorflow and wrote a program that uses CNNs to determine game results for a given tic-tac-toe board. Its inputs and outputs are -
Input - An array of 9 elements that represents the board (0=empty, 1='X', 2='O')
Output - An array of 3 elements ([1,0,0]=draw, [0,1,0]=player1 won, [0,0,1]=player2 won).
Below is the tensorflow graph part of the program (it is a modified version of tensorflow tutorial at - https://www.tensorflow.org/get_started/mnist/pros).
baseFeatureSize = 64
x = tf.placeholder(tf.float32, shape=[None, 9])
x_image = tf.reshape(x, [-1, 3, 3, 1])
W_conv1 = nn_utils.weight_variable([2, 2, 1, baseFeatureSize])
b_conv1 = nn_utils.bias_variable([baseFeatureSize])
h_conv1 = tf.nn.relu(nn_utils.conv2d(x_image, W_conv1) + b_conv1)
W_conv2 = nn_utils.weight_variable([2, 2, baseFeatureSize, baseFeatureSize * 2])
b_conv2 = nn_utils.bias_variable([baseFeatureSize * 2])
h_conv2 = tf.nn.relu(nn_utils.conv2d(h_conv1, W_conv2) + b_conv2)
W_fc1 = nn_utils.weight_variable([3 * 3 * baseFeatureSize * 2, baseFeatureSize * 4])
b_fc1 = nn_utils.bias_variable([baseFeatureSize * 4])
h_pool2_flat = tf.reshape(h_conv2, [-1, 3 * 3 * baseFeatureSize * 2])
h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)
keep_prob = tf.placeholder(tf.float32)
h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)
W_fc2 = nn_utils.weight_variable([baseFeatureSize * 4, 3])
b_fc2 = nn_utils.bias_variable([3])
y_ = tf.placeholder(tf.float32, shape=[None, 3])
y_conv = tf.matmul(h_fc1_drop, W_fc2) + b_fc2
cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y_conv))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
correct_prediction = tf.equal(tf.argmax(y_conv,1), tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
def conv2d(x, W): return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
def max_pool_2x2(x): return tf.nn.max_pool(x, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
def weight_variable(shape):
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial)
def bias_variable(shape):
initial = tf.constant(0.1, shape=shape)
return tf.Variable(initial)
Questions - The above model works well but requires to train with over 700k inputs to get output prediction accuracy up to 80%. But the total possible permutations of the game board are less than 300K. If I feed just all of the unique permutations, then the accuracy worsens.
Is it normal for CNNs to require training data size to be much larger than all possible permutations of the inputs? I'm assuming this is not true or else they'd not be able to play chess and go etc, so what am I doing wrong?
Based on your experience, how big a training sample size would typically make sense for this setup?