1
$\begingroup$

I think in writing we often refer to the part of the neural networks that tell us about the data are the parameters when we really mean the weights, is this correct? Would the sentence...

Better understanding parameters of the neural network after training on bird migration data can allow us to comprehend the behavior of these animals.

...be better phrased as...

Better understanding the weights of the neural network after training on bird migration data can allow us to comprehend the behavior of these animals.

$\endgroup$
  • $\begingroup$ Weights are one of the parameters, there are lot others which are of concern like deepness, hidden units, proper initialising, activations etc... $\endgroup$ – Aditya Jun 4 '18 at 16:08
  • $\begingroup$ @Aditya would other parameters besides the weights help you better understand the underlying data? $\endgroup$ – jchaykow Jun 4 '18 at 16:51
  • $\begingroup$ I don't have expertise in NN, but as far as I have read about them in general people except researchers don't care much about the weights, whether they are too big or too small, what we care is that it shouldn't overfit as it's a piece of cake walk to make them overfit, when they don't do so we can be in a position to think that it's performance is around the expectations... $\endgroup$ – Aditya Jun 4 '18 at 17:17
  • $\begingroup$ @jchaykow check the last part of my answer $\endgroup$ – pcko1 Jun 4 '18 at 17:59
2
$\begingroup$

That's partially right.

Apart from the weights (and bias) you also have the so called hyper-parameters. Depending on your architecture, you may have different ones. For a simple feedforward neural network, the hyperparameters are:

  1. number of neurons
  2. number of layers
  3. learning rate eta (η)
  4. regularization penalty lambda (λ)
  5. momentum
  6. number of epochs
  7. batch size
  8. dropout

etc

These hyperparameters are (usually) user-tuned. The weights (and bias) are self-trained by the network, based on optimization. Keep in mind that hyperparameters like the number of neurons or layers are immediately connected with the underlying model of your data, therefore they are an index of your dataset's complexity.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.