Questions tagged [finite-precision]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
1answer
167 views

FP16, FP32 - what is it all about? or is it just Bitsize for Float-Values (Python)

What is it all about FP16, FP32 in Python? My potential Business Partner and I are building a Deep Learning Setup for working with time series. He came up with "FP16 and FP32" while finding a GPU. It ...
0
votes
0answers
102 views

Keras custom metrics - MAP and MRR

I am trying to build a LSTM model in keras where I have one question with 10 answers but only ONE among them is correct. So basically im tring to build a 10 class classification problem. As most of ...
1
vote
1answer
11 views

Truncating float/doubles for reproducibility

I deploy machine learning models (typically GPU) to a variety of environments. I work sort of at the edge of ML R&D and devops, so I am really big into reproducibility, and one thing that drives ...
0
votes
0answers
61 views

Tensorflow using float64 tensor

I have a feed-forward neural network with a customized cost function. Since my cost function has an exponential component, I need to handle very large numbers. By default, tensorflow uses float32 ...
8
votes
2answers
4k views

Why is finite precision a problem in machine learning?

Can you explain what is finite precision? Why is finite precision a problem in machine learning?