3
$\begingroup$

I’m facing some (RAM) memory issues to train a neural network.

I have an input array consisting of grayscale images encoded as the type numpy.uint8 (therefore the whole range 0-255 can be covered). When feeding the data to the network, I’m supposed to normalize the values into the 0-1 range, making them now floating points (numpy.float64 or 32). However, this conversion makes my data 8x(or 4x) bigger, which my RAM memory can’t handle.

Would there be a way I could overcome this issue?

Thanks !

Improve this question
$\endgroup$
5
  • $\begingroup$ Are you loading the entire dataset into memory all at once? Could you instead stream the data from disk in batches? $\endgroup$
    – zachdj
    Oct 1, 2019 at 13:21
  • $\begingroup$ @zachdj I’m not sure on how to do this. From all the code examples I looked into Keras documentation, the training array is loaded at once, and the batch size is configured by the model.fit function. $\endgroup$
    – L. B.
    Oct 1, 2019 at 13:48
  • 1
    $\begingroup$ Instead of fit method, try fit_generator and generate images for every batch. Put an effort and I can tell u you will get rid of this RAM issues. $\endgroup$
    – Kiritee Gak
    Oct 1, 2019 at 14:39
  • 1
    $\begingroup$ Here's a tutorial on data generators in Keras: stanford.edu/~shervine/blog/… $\endgroup$
    – zachdj
    Oct 1, 2019 at 15:48
  • $\begingroup$ Thanks a lot KiriteeGak and zachdj. I’ll read about these data generators. $\endgroup$
    – L. B.
    Oct 1, 2019 at 17:28

1 Answer 1

Reset to default
1
$\begingroup$

After normalization, cast the numbers to float16 or bfloat16. Those are the smallest floats available in Keras.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.