1
$\begingroup$

I've got a problem trying to use generators in Keras. I've got TensorFlow 2.1 with Python 3.7. I'm using the Keras version bundled with TensorFlow

I have defined a class for my generator derived from tf.keras.models.Sequential:

class RandomFramesFromPathsToVideos(tf.keras.models.Sequential):
    def __init__(self, x_set, y_set, number_of_videos_per_batch, ort_session, frames_per_video=25, type_of_frame='cropped_frame'):
        self.x, self.y = x_set, y_set
        self.batch_size = number_of_videos_per_batch * frames_per_video
        self.number_of_videos_per_batch = number_of_videos_per_batch
        self.frames_per_video = frames_per_video
        self.type_of_frame = type_of_frame
        self.ort_session = ort_session



    def __len__(self):
        return int(np.ceil(len(self.x) / float(self.number_of_videos_per_batch)))

    def __getitem__(self, idx):

        bla, bla, bla...


        return (np.array(devol_x, dtype=np.float32), np.array(devol_y, dtype=np.float32))

and it works since I'm perfectly able to train my model item by item using a crafted loop like this:

ds_train = RandomFramesFromPathsToVideos(x_set = set_of_paths, y_set = set_of_labels, 
    number_of_videos_per_batch=4,
    ort_session = ort_session)

for i in range(0:len(ds_train)):
    (x,y)=ds_train.__getitem__(i)
    model.fit(x,y)

but when I try to use the generator as it should be used:

model.fit_generator(ds_train)

(which is now equivalent in TF2.1 to model.fit(ds_train), I know), I get the error:

ValueError: Failed to find data adapter that can handle input: <class 'functions.RandomFramesFromPathsToVideos'>, <class 'NoneType'>

I've tracked the error up to this point in internal code of TensorFlow:

def select_data_adapter(x, y):
  """Selects a data adapter than can handle a given x and y."""
  adapter_cls = [cls for cls in ALL_ADAPTER_CLS if cls.can_handle(x, y)]
  if not adapter_cls:
    # TODO(scottzhu): This should be a less implementation-specific error.
    raise ValueError(
        "Failed to find data adapter that can handle "
        "input: {}, {}".format(
            _type_name(x), _type_name(y)))
  elif len(adapter_cls) > 1:
    raise RuntimeError(
        "Data adapters should be mutually exclusive for "
        "handling inputs. Found multiple adapters {} to handle "
        "input: {}, {}".format(
            adapter_cls, _type_name(x), _type_name(y)))
  return adapter_cls[0]

The contents of ALL_ADAPTER_CLS is:

result = {list: 7} [<class 'tensorflow.python.keras.engine.data_adapter.ListsOfScalarsDataAdapter'>, <class 'tensorflow.python.keras.engine.data_adapter.TensorLikeDataAdapter'>, <class 'tensorflow.python.keras.engine.data_adapter.GenericArrayLikeDataAdapter'>, <class 'tensor
 0 = {ABCMeta} <class 'tensorflow.python.keras.engine.data_adapter.ListsOfScalarsDataAdapter'>
 1 = {ABCMeta} <class 'tensorflow.python.keras.engine.data_adapter.TensorLikeDataAdapter'>
 2 = {ABCMeta} <class 'tensorflow.python.keras.engine.data_adapter.GenericArrayLikeDataAdapter'>
 3 = {ABCMeta} <class 'tensorflow.python.keras.engine.data_adapter.DatasetAdapter'>
 4 = {ABCMeta} <class 'tensorflow.python.keras.engine.data_adapter.GeneratorDataAdapter'>
 5 = {ABCMeta} <class 'tensorflow.python.keras.engine.data_adapter.KerasSequenceAdapter'>
 6 = {ABCMeta} <class 'tensorflow.python.keras.engine.data_adapter.CompositeTensorDataAdapter'>
 __len__ = {int} 7

I don't understand why my generator doesn't work. Has anyone any idea about what I'm doing wrong?. My dataset is a huge bunch of videos and is impossible to fit in memory by any means. Also, I'm doing non standard transformations like locating and cropping elements that appear in the frames so I can't use an standard generator that may be already implemented in Keras.

Thanks a lot

$\endgroup$

1 Answer 1

1
$\begingroup$

I was following the instructions on a tutorial but it seems that there was a mistake or that the internal code of TensorFlow has changed since it was written. I was defining my class like this:

class DataGenerator(tf.keras.utils.Sequence):

and later on I tried also this:

class DataGenerator(tf.keras.models.Sequential):

(which, by the way, is completely different and is totally wrong)

Inspecting the code for some of the adapters, I saw this:

class KerasSequenceAdapter(GeneratorDataAdapter):
  """Adapter that handles `keras.utils.Sequence`."""

  @staticmethod
  def can_handle(x, y=None):
    return isinstance(x, data_utils.Sequence)

and also for my other suspect adapter:

class GeneratorDataAdapter(DataAdapter):
  """Adapter that handles python generators and iterators."""

  @staticmethod
  def can_handle(x, y=None):
    return ((hasattr(x, "__next__") or hasattr(x, "next"))
            and hasattr(x, "__iter__")
            and not isinstance(x, data_utils.Sequence))

where data_utils is defined as:

from tensorflow.python.keras.utils import data_utils

So, I changed the definition of my class to this:

from tensorflow.python.keras.utils import data_utils
class RandomFramesFromPathsToVideos(data_utils.Sequence):

and now it works.

I've got, so, two options. Using a subclass from data_utils.Sequence is one of them. In that case, we are talking about a "Keras generator" (handled in TensorFlow by KerasSequenceAdapter) in which you need to define two methods: __len__ and __getitem__

The other one is building a new class NOT DERIVED from data_utils.Sequence, and defining the methods __iter__ and __next__ (or simply next). In that case we are defining an standard Python generator, which will be handled by GeneratorDataAdapter inside TensorFlow

Please, bear in mind that a Keras generator is not the same thing as a Python generator. At first, that confused me a lot.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.