According to: Introducing native PyTorch automatic mixed precision for faster training on NVIDIA GPUs

It's better to use AMP (with 16 Floating Point) due to:

  1. Shorter training time.
  2. Lower memory requirements, enabling larger batch sizes, larger models, or larger inputs.
  • So is there a reason not to work with FP16 ?
  • Which models / datasets / solutions we will need to use FP32 and not FP16 ?
  • Can I find an example in kaggle which we must use FP32 and not FP16 ?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.