1
$\begingroup$

I have a dataset that it's classes arranged in the following way:

/dataset/train/images/class1/

/dataset/train/images/class2/
.
.
.
/dataset/train/images/classN/

Does anyone have any idea how to store data in a train_ds variable with the help of TFDS library?

$\endgroup$
1
$\begingroup$

This is a common folder structure for Image Classification, so common many libraries have a dataset class for it (torchvision, fastai, tfds), usually called ImageFolder.

In the case of TFDS this is implemented in the tfds.folder_dataset.ImageFolder:

tfds.folder_dataset.ImageFolder(
    root_dir: str,
    *,
    shape: Optional[type_utils.Shape] = None,
    dtype: Optional[tf.DType] = None
)

As your folder already has the expected format, i.e.:

  split_name/  # Ex: 'train'
    label1/  # Ex: 'airplane' or '0015'
      xxx.png
      xxy.png
      xxz.png
    label2/
      xxx.png
      xxy.png
      xxz.png
  split_name/  # Ex: 'test'

You could just instantiate it like this:

train_ds = tfds.folder_dataset.ImageFolder(
    root_dir = "/dataset/", # Note that this is a absolute path, you should use "./dataset/" or "dataset/" or "<current_working_dir_full_path>/dataset/" if that is the case.
)

# If you want a tensorflow.data.Dataset
train_ds = train_ds.as_dataset(**args)

Suported args for as_dataset method are:

(
    split: Optional[Union[str, tfds.core.ReadInstruction]] = None,
    batch_size: tfds.typing.Dim = None,
    shuffle_files: bool = False,
    decoders: Optional[TreeDict[decode.Decoder]] = None,
    read_config: Optional[tfds.ReadConfig] = None,
    as_supervised: bool = False
)

You could refer to documentation for details.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.