I am trying to build and train a multilayer perceptron neural network that correctly predicts what president won in what county for the first time. I have the following information for training data.
Total population Median age % BachelorsDeg or higher Unemployment rate Per capita income Total households Average household size % Owner occupied housing % Renter occupied housing % Vacant housing Median home value Population growth House hold growth Per capita income growth Winner
That's 14 columns of training data and the 15th column is what the output should be.
I am trying to use Keras to build a multilayer perceptron neural network, but I need some help understanding a few properties and the pros of cons of choosing different options for these properties.
- ACTIVATION FUNCTION
I know my first step is to come up with an activation function. I always studied neural networks used sigmoid activation functions. Is a sigmoid activation function the best? How do you know which one to use? Keras additionally gives the options of using a softmax, softplus, relu, tanh, linear, or hard_sigmoid activation function. I'm okay with using whatever, but I just want to be able to understand why and the pros and cons.
- PROBABILITY INITIALIZAIONS
I know initializations define the probability distribution used to set the initial random weights of Keras layers. The options Keras gives are uniform lecun_uniform, normal, identity, orthogonal, zero, glorot_normal, glorot_uniform, he_normal, and he_uniform. How does my selection here impact my end result or model? Shouldn't it not matter because we are "training" whatever random model we start with and come up with a more optimal weighting of the layers anyways?