4
$\begingroup$

In case of image classification with 0-255 pixel values of integer type, is it necessary/recommended to convert the values into float with Neural Networks? This conversion seems me unnecessary (moreover, float slows down matrix operations) but several implementations follow this practice. Why?

$\endgroup$
3
$\begingroup$

Technically with most languages you could pass in integer features for the input layer, since the weights will be floats, and multiplying a float by an integer will give you a float. Also, you don't usually care about partial derivatives of the input data, so it doesn't matter that the values are discrete.

However:

  • For all weights and neuron activations, if you are using a method based on backpropagation for training updates, then you need a data type that approximates real numbers, so that you can apply fractional updates based on differentiation. Best weight values are often going to be fractional, non-whole numbers. Non-linearities such as sigmoid are also going to output floats. So after the input layer you have matrices of float values anyway. There is not much speed advantage multiplying integer matrix with float one (possibly even slightly slower, depending on type casting mechanism). So the input may as well be float.

  • In addition, for efficient training, the neural network inputs should be normalised to a specific roughly unit range (-1.0 to 1.0) or to mean 0, standard deviation 1.0. Both of these require float representation. If you have input data in 0-255 range - float or not - you will usually find the network will learn less effectively.

There are exceptions to these rules in some architectures and learning algorithms, where perhaps an integer-based input would work, but for the most common NN types, including MLP and "deep" feed-forward networks, it is simpler and easier to use float data type.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.