Skip to main content
Data Science

Return to Question

Clarify the point of the question
Source Link
user7226068
user7226068
  • 61
  • 3

I use a brute-force mechanism to determine optimal hidden layers/neurons by incrementing the layers/neurons by 1 up to some maximums and then picking the optimal counts from the best performing model. My question here is about the starting point for the hidden neuron count of this "brute-force" process. What are some good mechanisms for determining what neuron count to start at such that I am not likely to miss some minimum optimal count?

For example:

Say I have 8 inputs that are NOT linearly related. If I take (number of inputs: 8) + 1 (as some post suggest) for my starting point of incrementing the number of neurons within the layers, would I likely be missing the optimal neurons count as it may be less than 9 neurons?

The brute-force mechanism I am using sets a max number of neurons per hidden layer, so if I have 8 inputs, and I started with a range of 9 to 30 neurons per hidden layer, I would have 9 neurons in the first hidden layer which would increment to 30 and then roll back to 9 when starting to test with a second hidden layer:

Cycle 1 Hidden Layers = 1 Hidden Neurons = 9 ... Hidden Layers = 1 Hidden Neurons = 30

Cycle 2 Hidden Layers = 2 Hidden Neurons = 9 ... Hidden Layers = 2 Hidden Neurons = 30

Cycle 3 Hidden Layers = 3 Hidden Neurons = 9 ... Hidden Layers = 3 Hidden Neurons = 30

I admit this has inefficiencies as each layer may not need the same number of neurons, but the point of this post is to try and improve this mechanism by minimizing the number of layers/neurons combinations that need to be tested to determine the "optimal" counts.

EDIT:

So just to clarify the question is about determining what a good minimum starting point would be if using the brute-force mechanism outlined above regardless of data domain (hence the input + 1 example).

I use a brute-force mechanism to determine optimal hidden layers/neurons by incrementing the layers/neurons by 1 up to some maximums and then picking the optimal counts from the best performing model. My question here is about the starting point for the hidden neuron count of this "brute-force" process. What are some good mechanisms for determining what neuron count to start at such that I am not likely to miss some minimum optimal count?

For example:

Say I have 8 inputs that are NOT linearly related. If I take (number of inputs: 8) + 1 (as some post suggest) for my starting point of incrementing the number of neurons within the layers, would I likely be missing the optimal neurons count as it may be less than 9 neurons?

The brute-force mechanism I am using sets a max number of neurons per hidden layer, so if I have 8 inputs, and I started with a range of 9 to 30 neurons per hidden layer, I would have 9 neurons in the first hidden layer which would increment to 30 and then roll back to 9 when starting to test with a second hidden layer:

Cycle 1 Hidden Layers = 1 Hidden Neurons = 9 ... Hidden Layers = 1 Hidden Neurons = 30

Cycle 2 Hidden Layers = 2 Hidden Neurons = 9 ... Hidden Layers = 2 Hidden Neurons = 30

Cycle 3 Hidden Layers = 3 Hidden Neurons = 9 ... Hidden Layers = 3 Hidden Neurons = 30

I admit this has inefficiencies as each layer may not need the same number of neurons, but the point of this post is to try and improve this mechanism by minimizing the number of layers/neurons combinations that need to be tested to determine the "optimal" counts.

I use a brute-force mechanism to determine optimal hidden layers/neurons by incrementing the layers/neurons by 1 up to some maximums and then picking the optimal counts from the best performing model. My question here is about the starting point for the hidden neuron count of this "brute-force" process. What are some good mechanisms for determining what neuron count to start at such that I am not likely to miss some minimum optimal count?

For example:

Say I have 8 inputs that are NOT linearly related. If I take (number of inputs: 8) + 1 (as some post suggest) for my starting point of incrementing the number of neurons within the layers, would I likely be missing the optimal neurons count as it may be less than 9 neurons?

The brute-force mechanism I am using sets a max number of neurons per hidden layer, so if I have 8 inputs, and I started with a range of 9 to 30 neurons per hidden layer, I would have 9 neurons in the first hidden layer which would increment to 30 and then roll back to 9 when starting to test with a second hidden layer:

Cycle 1 Hidden Layers = 1 Hidden Neurons = 9 ... Hidden Layers = 1 Hidden Neurons = 30

Cycle 2 Hidden Layers = 2 Hidden Neurons = 9 ... Hidden Layers = 2 Hidden Neurons = 30

Cycle 3 Hidden Layers = 3 Hidden Neurons = 9 ... Hidden Layers = 3 Hidden Neurons = 30

I admit this has inefficiencies as each layer may not need the same number of neurons, but the point of this post is to try and improve this mechanism by minimizing the number of layers/neurons combinations that need to be tested to determine the "optimal" counts.

EDIT:

So just to clarify the question is about determining what a good minimum starting point would be if using the brute-force mechanism outlined above regardless of data domain (hence the input + 1 example).

Source Link
user7226068
user7226068
  • 61
  • 3

Minimum Neurons in Neural Network

I use a brute-force mechanism to determine optimal hidden layers/neurons by incrementing the layers/neurons by 1 up to some maximums and then picking the optimal counts from the best performing model. My question here is about the starting point for the hidden neuron count of this "brute-force" process. What are some good mechanisms for determining what neuron count to start at such that I am not likely to miss some minimum optimal count?

For example:

Say I have 8 inputs that are NOT linearly related. If I take (number of inputs: 8) + 1 (as some post suggest) for my starting point of incrementing the number of neurons within the layers, would I likely be missing the optimal neurons count as it may be less than 9 neurons?

The brute-force mechanism I am using sets a max number of neurons per hidden layer, so if I have 8 inputs, and I started with a range of 9 to 30 neurons per hidden layer, I would have 9 neurons in the first hidden layer which would increment to 30 and then roll back to 9 when starting to test with a second hidden layer:

Cycle 1 Hidden Layers = 1 Hidden Neurons = 9 ... Hidden Layers = 1 Hidden Neurons = 30

Cycle 2 Hidden Layers = 2 Hidden Neurons = 9 ... Hidden Layers = 2 Hidden Neurons = 30

Cycle 3 Hidden Layers = 3 Hidden Neurons = 9 ... Hidden Layers = 3 Hidden Neurons = 30

I admit this has inefficiencies as each layer may not need the same number of neurons, but the point of this post is to try and improve this mechanism by minimizing the number of layers/neurons combinations that need to be tested to determine the "optimal" counts.