Questions tagged [data-augmentation]
The data-augmentation tag has no usage guidance.
96
questions
0
votes
1answer
18 views
How much data augmentation is required on an imbalanced dataset?
Imagine I have a dataset with positive and negative sentences, and I need to train a transformer (Like BERT) to do the binary classification. The problem is that there are 100 negative sentences and ...
0
votes
0answers
15 views
Data augmentation within epochs vs across epochs
Usually in deep learning data augmentation is applied by creating a new augmented version of each training sample for each epoch. Therefore the amount of training samples for each epoch stays the same ...
0
votes
0answers
19 views
How do I perform permutation of columns in a pandas dataframe in group of threes
I am relatively new to this platform and python in gerenal. I have got a data frame as follows and I want to implement data augmentation algorithm by permutation. The numbers in the data frame are XYZ ...
0
votes
0answers
5 views
save reconstructed data points from variational autoencoder as original MNIST
I have a VAE implementation that generates images from the latent distribution. I want to save those "images" as we have in the original dataset. For example, my VAE generates a data point, ...
0
votes
1answer
18 views
Batch normalization for multiple datasets?
I am working on a task of generating synthetic data to help the training of my model. This means that the training is performed on synthetic + real data, and tested on real data.
I was told that batch ...
1
vote
0answers
16 views
Specify torchvision transforms depending on the properties of an image and a mask
I have a dataset 1000 of images and corresponding segmentation masks from dermatologists. The images come in different sizes (as low as 400x600 and as large as 4Kx4K). 95% of image pixels are not ...
0
votes
0answers
21 views
How can I get dark, median-brightness and bright distribution histograms from an image?
I'm trying to replicate an augmentation technique used in this paper. This is how they explain the procedure:
[...] the augmentation technique was used to adjust the histogram because the intensity ...
0
votes
1answer
59 views
What is the scope of Keras' ImageDataGenerator.flow_from_dataframe seed parameter?
I've been working on a U-Net model using training images stored on my local drive. To load these I have been using Keras' ImageDataGenerator.flow_from_dataframe ...
0
votes
0answers
24 views
SMOTE for Image regression?
Can you use something like SMOTE for an image regression task, where the target value is very skewed and imbalanced? I already tried using classic augmentation techniques like flipping, cropping, etc. ...
1
vote
0answers
14 views
Working on an image classification project (microscopic images) , have some doubts [closed]
Currently, I am working on an image classification project. The data set contains very high resolution images taken via an electron microscope. Hence, I have few and limited instances.
I have done EDA ...
3
votes
1answer
212 views
Why does adding data augmentation decrease training accuracy a tiny bit?
Before data augmentation, my model clearly overfits and hits a 100% training accuracy and a 52% validation accuracy. When only adding data augmentation with Keras, as a regularization technique, it ...
0
votes
1answer
43 views
Does Mixup requires two loss functions?
I created a neural network with multi-label classification using MSE.
Now, I would like to use Mixup. Do I need two loss functions (for each target one) or is the result the same if I just combine the ...
1
vote
1answer
54 views
How to mantain the nested structure of a tf.dataset after applying map?
I'm creating a tf.dataset object containing 2 images as inputs and a mask as target. All of them are 3D in grayscale. After applying a custom map, the shape of the object changes from ...
0
votes
1answer
83 views
How to implement random cropping during training?
I'm developing a U-net like model which segments the damaged tissue of the brain between two time-points in Multiple Sclerosis patients. The model is given the baseline and follow-up images as x and ...
0
votes
0answers
30 views
Data augmentation for tabular data in a multi label classification task
The task at hand is to predict the future lab values for a patient (1 if abnormal and 0 if normal) using the previous numerical data. It is a multi-label, multi-class time series classification task. ...
0
votes
1answer
99 views
Rescale parameter in data augmentation
I'm a little bit lost about the rescale parameter in the ImageDatagenerator function.
I know that the rescale argument by itself does not augment my data and that ...
0
votes
0answers
25 views
Balancing on the particular imbalanced classes of image dataset
I have a dataset that has 12 classes in the base directory. However, these 12 classes consist of several amounts of Images. The number of images of 12 classes is inconsistent therefore its impacts the ...
0
votes
1answer
283 views
How many images are generated when ImageDataGenerator is used, and when data augmentation is included as a part of the model?
Is there any way to know the number of images generated by the ImageDataGenerator class and loading data using flow_from_directory method? I searched everywhere for the same but couldn't find anything ...
2
votes
0answers
47 views
Is it possible to increase the number of images of one class using data augmentation, which is not applied on the other class, in the same dataset?
I have 2 classes for my image classification problem, say class A and class B, and I am using tensorflow and keras for the same. One of them have around 5K images while the other have just around 2K ...
0
votes
0answers
6 views
How do you compare outputs from different augmentations for consistency training?
I'm trying to figure out how papers using consistency train on unsupervised data, and I'm stuck on how outputs from different augmentations are compared. Since the augmentations are transformations, I ...
0
votes
0answers
25 views
Constraining a Deep Neural Network based on a priori knowledge of a real world system
I am new to this field and to StackExchage, so I guess I'll start by saying hello!
I am building a deep neural network to model a physical system which takes a set of inputs based on real-world ...
2
votes
2answers
186 views
What is the use of using width/height shift in data augmentation?
I'm not sure to understand the use of augmentation data using width shift and height shift.
Say I have limited image data, and I want to create new data using Keras' ImageDataGenerator. To classify ...
0
votes
0answers
20 views
Should I resize my object localization dataset in order to train a fully convolutional network?
I want to create a fully-convolutional neural net that trains on wider face datasets in order to draw bounding box around faces. The dataset is highly diverse in the image sizes. So my question is, ...
1
vote
1answer
31 views
Is it better to augment both training and validation sets or just training set?
Is it better to augment data both training and validation sets or just the training set in order to achieve the best accuracy possible on a convolutional neural network? why?
0
votes
1answer
20 views
Textbook definition of Data augmentation
I'm trying to find a textbook definition of Data augmentation. We all know what it is, but I'm having a hard time finding a reliable source which I can cite for my paper:
The Wikipedia article on ...
1
vote
1answer
152 views
First perform data augmentation or normalization?
Should I first perform data augmentation or normalization in deep learning? I am mainly interested in 2D and 3D input data. In tutorials that I have seen so far the data augmentation always comes ...
1
vote
2answers
103 views
Why should I use data augmentation as Keras layer
I've seen in several Tensorflow/Keras tutorials that data augmentation functions are added as keras layers. When I converted my Keras Python model (for production purpose) to TensorflowJS I faced the ...
0
votes
0answers
17 views
ImageDataGenerator augmenting only one image in training set
My datset consists of images in 10 classes. I am using 'ImageDataGenerator' to augment my training data using 'flow_from_directory' with training directory containing subdirectories of classes, each ...
2
votes
1answer
801 views
What exactly are the data augmentation experimental Keras' layers doing?
From what I gathered, data augmentation consists in increasing your number of instances in your dataset by applying some transfromations. Let's say I want to classify images. If I apply a random ...
0
votes
0answers
14 views
Modify keras_unet.utils.get_augmented to read images from disk
I want to train a cnn model on a large dataset (10K images and masks).
Currently, I am reading data in batches of BATCH_SIZE = 500 images, augment with
...
0
votes
1answer
15 views
Are GPTs close to real intelligence or just another Data In -Data Out -Data Permutation and Combinations?
Use cases and Solutions surrounding GPT's have taken NLP world with storm and started the GPT-Best vs GPT- Not So Best war on the internet. There are solutions been derived from API's provided by HF. ...
1
vote
1answer
36 views
Is there any possibility to apply deep dreaming in data augmentation?
I looked into the deep dreaming concepts and feel like this has the potential for data generation. But i'm not sure how possible this concept is. Any thoughts regarding this?
0
votes
0answers
190 views
How to change voice features in python without affecting speech/language features?
I am trying to build a CNN model which should be able to identify the language being spoken in an audio file. I have extracted the MFCC matrix (for 13 coefficients) for each audio file and trained it....
1
vote
2answers
99 views
Problem of continuous training - Supervised learning
I am sure this is a most common problem, but would like to know by experts on how to tackle it. Note that, I mostly deal with textual data (NLP problems).
When a supervised learning model is created, ...
4
votes
2answers
653 views
Why are results without Transfer Learning better than with Transfer Learning?
I developed a neural network for license plate recognition and used the EfficientNet architecture (https://keras.io/api/applications/efficientnet/#efficientnetb0-function) with and without pretrained ...
1
vote
0answers
181 views
Data augmentation for recommendation systems
I have a user-item matrix that I use to train a denoising autoencoder to predict the top-k items to recommend to the different users.
The idea is to corrupt the matrix by erasing a percentage ...
1
vote
1answer
56 views
Training with less data
Most problem with machine learning projects I have faced is the lack of data. The samples available are enough to disqualify rule based approach but not enough for a neural network to train.
For ...
0
votes
2answers
153 views
How to train a keras model on both original and augmented data from ImageDataGenerator?
I have a dataset that contains about 87000 images in a directory, with each class in a separate subfolder. I've tried the class ImageDataGenerator() and the ...
1
vote
2answers
305 views
Should augmentation also be performed on the validation set when the dataset is imbalanced?
I am training a CNN on images (2 classes) and I have an imbalanced dataset (1:7 ratio). I am trying to tackle this by performing offline image augmentation. Should I perform augmentation also on the ...
0
votes
1answer
37 views
What's the difference between keras api augmentation and data augmentation definition?
The augmentation definition is increasing the number of images by using rotation, crop and flip to avoid overfitting. The keras API apply augmentation but no increasing the number of image. What keras ...
2
votes
1answer
164 views
How to do data augmentation for Machine Learning on statistical data?
I am training machine learning classification model. I have data in csv format with lets say 5 features(or columns) and 100 such observations(or rows).
I want to add more similar data to improve my ...
1
vote
0answers
126 views
What is the difference between domain randomization and data augmentation?
Domain randomization (https://arxiv.org/abs/1703.06907) is used to create a synthetic dataset with enough variance that it will encompass unseen real data, as just one variation.
I am trying to ...
0
votes
0answers
108 views
Problem with data augmentation only on train set!
So I want to augment my data but unfortunately, I don't have an individual folder for train and validation set, and of course, I want to only augment the train set but if I use this:
...
0
votes
1answer
1k views
Data augmentation for multiple output heads in Keras
I have a transfer learning based two output classification problem. So, accordingly, I have formatted my data to have X_train as a ...
1
vote
1answer
100 views
How can I improve my model on a very very small dataset?
I am starting as a PhD student and we want to find appropriate materials (with certain qualities) from basic chemical properties like charge, etc. There are a lot of models and datasets in similar ...
1
vote
1answer
54 views
After performing data Augmentation on tf.data.Dataset, should i MERGE it with original tf.data.Dataset?
Kind of a silly question, but I read that data Augmentation can be used in order to solve problem of small datasets. In my case, I've got a dataset with 5 different classes and around 2k examples per ...
1
vote
0answers
105 views
Flow_from_directory() won't load any image
i'm trying to load images for training a CNN using keras flow_from_directorY(), but seems that is having hard times finding the images. i'm running the code on google colab
...
1
vote
0answers
1k views
ValueError: could not convert string to float: 'Nor967.jpg'
Whenever I try to use the data augmentation ImageDataGenerator I'm getting an error like could not convert string to float. Here is my code.
...
0
votes
1answer
99 views
Why do we use different image processing at train and test time for an image classification tasl
I would like to ask why we use different image processing at train and test for an image classification task. For example, this pytorch tutorial uses RandomResizedCrop at train time and then Resize + ...
1
vote
1answer
187 views
Using MultiLabelBinarizer for SMOTE
This is my first NLP project. I'm trying to use SMOTE for a classifier with 14 classes. I need to convert the classes into an array before using SMOTE. I tried using MultiLinearBinarizer but it does ...
