3
$\begingroup$

Nvidia is planing to add hardware support for int8 operations to their titan card and target deep learning. I am trying to understood how its useful and what types of network will benefit from this.

I know that FP16 instead of FP32 is what should be useful for DL, but not sure how int8 could do. There are some research that you can train with full FP32 precision and then round it to one byte - but this does not speedup or reduce memory footprint for training.

$\endgroup$
  • $\begingroup$ Some papers I've come across (e.g., Deep Learning with Limited Numerical Precision) indicate that 8 bits are not enough for adequately training neural networks, but someone might devise a general approach that overturns this conclusion, so it's good for the hardware to be ahead of the software. Here's one attempt. Welcome to DataScience.SE! $\endgroup$ – Emre Jul 26 '16 at 17:48
2
$\begingroup$

Actually, recently people have been trying much lower precision in neural nets: 1-2-5 scheme (1 bit weights, 2 bit activations, and 5 bit gradients) seems to work well for easy datasets (MNIST and CIFAR-10). However, on ImageNet the results are significantly lower than those with full-precision (16 or 32 bit). To achieve state of the art, convnets don't need more than 16 bits for training, but current RNNs might need more. For inference, on ImageNet, 4-5 bit weights (stochastically rounded from full precision) should be enough.

$\endgroup$
0
$\begingroup$

The 8-bit integer could be enough to train the neural network. This link says that Intel successfully trained ResNet-50 with 8-bit integer only, using some techniques.

While this link is about post-training quantization. It aims to train in fp32 (or fp16), and make inference on uint8. It is already being used in Tensorflow Lite, and does not need any specific techniques.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.