I am building a Convolutional Neural Network with Keras. I want to use my own dataset. This dataset consists of matrices of specified size and filled with 0's which contain a submatrix of specified size and filled with 1's. The submatrix is optional and the goal is to train the NN to predict whether a matrix contains the submatrix or whether it doesnt. To make it more difficult to detect, I am adding various types of noise to the matrices.
Now I want to create a CNN which can classify these images. The easiest approach so far was to use
test_set = test_datagen.flow_from_directory('dataset/training_set', target_size = (imageSize, imageSize), batch_size = batchSize, class_mode = 'binary')
And then I fill the directory
training_set with 1000's of images. This works well, but has one major problem:
It is really inefficient, as I'm first creating images, saving them on a hard-dsik, then loading them again. These are a lot of unecessary steps, considering that keras transforms the images back into matrices (or tensors, idk), and my images were initially matrices.
Thus I was thinking about improving the efficiency by having the matrices in a
.csv or even having my
matrixgenerator.py "feed" a list or array directly into keras. I did some research and found the method flow from Keras which specifies as parameter an input matrix.
Unfortunately this is where I am stuck. I do not understand how to get my matrices into the form keras wants. I also don't know the exact shape the matrices and labels need to have to be able to use the flow method.
It would really help if someone could explain me how to use the flow method/tell me where I find more information on it, or give me a better option to feed my dataset into a keras CNN. Thank you for your help. I'm also ok with using a lower-level tensorflow implementation as long as I understand it.