I'm concerned that I'm attempting the impossible with my neural network. This is the scenario:
I have a 2D square world. In it, I create five circles of random size and position. I then classify one of them as the correct answer, based on the following rules:
- If any circle's radius is > THRESHOLD, I choose the largest circle
- Otherwise, I choose the circle with the origin nearest the center
I send the inputs as serial coordinates, like this: [X0, Y0, RADIUS0, X1, Y1, RADIUS1, ...].
The output is a one-hot array, e.g. [0, 0, 1, 0, 0].
I've modeled this in TensorFlow without success. My best scoring result appears to always choose the largest circle, ignoring the else clause of the arbitrary rule.
Am I fundamentally misunderstanding the capabilities of neural networks? I've tried many (many) different configurations (layer counts, node counts, activation functions ... you name it). All of my networks have been feed-forward, so far.
Thanks in advance for any insight!
Here are some details of my network and data:
- I have tried with up to 500k cases. I separate 10% for generalization checks after training, and train on the remaining 90% with a 50/50 validation split.
- I've tried with the test data weighted 75% toward rule A, 50/50, and 75% toward ruleB.
- I've tried 0-10 hidden layers, and neuron counts from 2 to 256 (each hidden layer gets the same number of neurons).
- I change the number epochs as time allows, but generally it's 10-100. My longest runs have been several hours (with giant case numbers, and dropouts to prevent overfitting).
- I've used batch sizes of 1-50.
- I've tried learning rates of 0.0001 - 0.1.
- I'm currently using ReLU activation, initializing bias to const(0.1) and kernel w/ heNormal. I have tried several other approaches for all three.
- I standardize the inputs to center on zero w/ variance of one.
- The loss function is categoricalCrossentropy.
- The optimizer is Adam.