Questions tagged [data-augmentation]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
18 views

Most efficient use of rare real input images - in training or validation set?

I want to do image segmentation with only very few realistic example images. Do I train on artificial data only and use the few real images as validation, thereby never directly learning from them at ...
user avatar
  • 1
0 votes
0 answers
8 views

what libraries are available to generate augmented (synthetic) data at a vector level?

For instance, I have an embedding and I wish to generate a sampling of vectors similar to a sample embedding that can then be used for training a model.
user avatar
  • 143
0 votes
0 answers
10 views

Data Augmentation Keras length of data

I'm confused when I add data augmentation should I get more data or the same data I tested my x_train length to confirm but I got the same length before augmentation and after augmentation is that ...
user avatar
  • 1
1 vote
0 answers
37 views

Can data augmentation techniques be misleading?

In an attempt to handle imbalance in data, especially in the case of extremely imbalanced data, can the various data augmentation techniques create some bias?
user avatar
  • 11
0 votes
0 answers
16 views

Is There Techniques for creating synthetic Data for Regression Problem i tried SMOTE and its variant but these are for classification problem

This is my data "Volume" is my Target variable and all other are Independent variables i just applied labelencoder on Area_categ , wind_direction_labelencod and on current _label_encode and ...
user avatar
2 votes
1 answer
27 views

Data augmentation in images

Suppose there is a ML network that takes grayscale images as the input. The images that I have are RGB images. So, instead of converting these RGB images to grayscale, I treat each individual colour ...
user avatar
1 vote
1 answer
21 views

Does synthetic data be over sampled as well?

I'm building a binary text classifier, the ratio between the positives and negatives is 1:100 (100 / 10000). By using back translation as an augmentation, I was able to get 400 more positives. Then I ...
user avatar
0 votes
0 answers
17 views

Is there any papers on adaptive discriminator augmentation in 3D?

I am really impressed with the results of ADA in action. Currently I work with 2D data (normal png images) but I would like to train StyleGAN2 + ADA in 3D space. Is there any papers/implementations ...
user avatar
0 votes
0 answers
19 views

How should I improve my CNN binary classification model from overfitting and underfitting [duplicate]

I am trying to do the cats & dogs classification problem, the problem is that my model is overfitting and I have tried all the techniques I know in order to solve but nothing is working such as ...
user avatar
0 votes
0 answers
19 views

Baseline model and transfer learning

I've tried to find any guidance on using transfer learning when building baseline models for ML projects (CNN in my case) but found no clues on good practices in the matter. My logic says that no ...
user avatar
1 vote
1 answer
88 views

Why we call Mix-up method is a data augmentation technique?

I am bit confused in the Mixup data augmentation technique, let me explain the problem briefly: What is Mixup For further detail you may refer to original paper . We double or quadruple the data ...
user avatar
  • 140
0 votes
0 answers
32 views

In a CNN architecture, is it possible to incorporate both class weights and data augmentation?

I'd like to conduct image classification using some CNN architectures, but the problem is that my classes are imbalanced, and each class has insufficient data. To solve this situation, I have a ...
user avatar
0 votes
0 answers
49 views

CutMix VS Mixup Data Augumentation method for end-to-end deep learning Traning

I am looking for arguments on which Data augmentation (Mixup VS CutMix) method would be preferable for Image data and Time-series classification data. As for as I know, both have the following ...
user avatar
  • 140
0 votes
0 answers
26 views

Augmentation for sound recognition of dog barks for CNNs

I am training CNNs to recognize dog barking, and for this I would like to augment the data sets I have (~30'000 10s clips with either barks, or no-barks in them). The straight forward idea was to mix ...
user avatar
1 vote
1 answer
74 views

ValueError: Error when checking input: expected time_distributed_6_input to have 5 dimensions, but got array with shape (32, 224, 224, 3) [closed]

I am trying to apply data augmentation to avoid overffiting in my CNN-LSTM image classification model. My training data has the shape: (1882, 1, 224, 224, 3) My ...
user avatar
  • 111
1 vote
1 answer
89 views

how does zoom out works in data augmentation?

how does zoom out works in data augmentation? I'm reading a doc on data augmentation in Keras ,and it says that randomZoom(0.2) zooms in and out by a factor in rage ...
user avatar
0 votes
2 answers
100 views

Removing outliers from a multi-dimensional dataset & Data augmentation

Removing the outliers of a single-dimensional data can be easily done by removing the points that are outside of the IQR range. But how should the process of detecting and removing outliers be done if ...
user avatar
1 vote
0 answers
28 views

Methods to combine datasets from different time periods

Consider a multivariate time series forecasting task where I have two datasets A and B. A goes from 1960 to 2020 and B goes from 2010 to 2020. There is a feature f ...
user avatar
  • 55
0 votes
2 answers
78 views

How much data augmentation is required on an imbalanced dataset?

Imagine I have a dataset with positive and negative sentences, and I need to train a transformer (Like BERT) to do the binary classification. The problem is that there are 100 negative sentences and ...
user avatar
0 votes
0 answers
80 views

Data augmentation within epochs vs across epochs

Usually in deep learning data augmentation is applied by creating a new augmented version of each training sample for each epoch. Therefore the amount of training samples for each epoch stays the same ...
user avatar
  • 101
0 votes
0 answers
466 views

How do I perform permutation of columns in a pandas dataframe in group of threes

I am relatively new to this platform and python in gerenal. I have got a data frame as follows and I want to implement data augmentation algorithm by permutation. The numbers in the data frame are XYZ ...
user avatar
  • 1
0 votes
0 answers
10 views

save reconstructed data points from variational autoencoder as original MNIST

I have a VAE implementation that generates images from the latent distribution. I want to save those "images" as we have in the original dataset. For example, my VAE generates a data point, ...
user avatar
0 votes
1 answer
32 views

Batch normalization for multiple datasets?

I am working on a task of generating synthetic data to help the training of my model. This means that the training is performed on synthetic + real data, and tested on real data. I was told that batch ...
user avatar
  • 111
1 vote
0 answers
35 views

Specify torchvision transforms depending on the properties of an image and a mask

I have a dataset 1000 of images and corresponding segmentation masks from dermatologists. The images come in different sizes (as low as 400x600 and as large as 4Kx4K). 95% of image pixels are not ...
user avatar
0 votes
0 answers
24 views

How can I get dark, median-brightness and bright distribution histograms from an image?

I'm trying to replicate an augmentation technique used in this paper. This is how they explain the procedure: [...] the augmentation technique was used to adjust the histogram because the intensity ...
user avatar
0 votes
1 answer
139 views

What is the scope of Keras' ImageDataGenerator.flow_from_dataframe seed parameter?

I've been working on a U-Net model using training images stored on my local drive. To load these I have been using Keras' ImageDataGenerator.flow_from_dataframe ...
user avatar
  • 111
1 vote
0 answers
16 views

Working on an image classification project (microscopic images) , have some doubts [closed]

Currently, I am working on an image classification project. The data set contains very high resolution images taken via an electron microscope. Hence, I have few and limited instances. I have done EDA ...
user avatar
  • 11
3 votes
1 answer
1k views

Why does adding data augmentation decrease training accuracy a tiny bit?

Before data augmentation, my model clearly overfits and hits a 100% training accuracy and a 52% validation accuracy. When only adding data augmentation with Keras, as a regularization technique, it ...
user avatar
  • 33
0 votes
1 answer
148 views

Does Mixup requires two loss functions?

I created a neural network with multi-label classification using MSE. Now, I would like to use Mixup. Do I need two loss functions (for each target one) or is the result the same if I just combine the ...
user avatar
1 vote
1 answer
236 views

How to mantain the nested structure of a tf.dataset after applying map?

I'm creating a tf.dataset object containing 2 images as inputs and a mask as target. All of them are 3D in grayscale. After applying a custom map, the shape of the object changes from ...
user avatar
0 votes
1 answer
193 views

How to implement random cropping during training?

I'm developing a U-net like model which segments the damaged tissue of the brain between two time-points in Multiple Sclerosis patients. The model is given the baseline and follow-up images as x and ...
user avatar
1 vote
1 answer
522 views

Rescale parameter in data augmentation

I'm a little bit lost about the rescale parameter in the ImageDatagenerator function. I know that the rescale argument by itself does not augment my data and that ...
user avatar
  • 113
0 votes
1 answer
840 views

How many images are generated when ImageDataGenerator is used, and when data augmentation is included as a part of the model?

Is there any way to know the number of images generated by the ImageDataGenerator class and loading data using flow_from_directory method? I searched everywhere for the same but couldn't find anything ...
user avatar
3 votes
0 answers
179 views

Is it possible to increase the number of images of one class using data augmentation, which is not applied on the other class, in the same dataset?

I have 2 classes for my image classification problem, say class A and class B, and I am using tensorflow and keras for the same. One of them have around 5K images while the other have just around 2K ...
user avatar
2 votes
2 answers
399 views

What is the use of using width/height shift in data augmentation?

I'm not sure to understand the use of augmentation data using width shift and height shift. Say I have limited image data, and I want to create new data using Keras' ImageDataGenerator. To classify ...
user avatar
0 votes
0 answers
24 views

Should I resize my object localization dataset in order to train a fully convolutional network?

I want to create a fully-convolutional neural net that trains on wider face datasets in order to draw bounding box around faces. The dataset is highly diverse in the image sizes. So my question is, ...
user avatar
1 vote
1 answer
51 views

Is it better to augment both training and validation sets or just training set?

Is it better to augment data both training and validation sets or just the training set in order to achieve the best accuracy possible on a convolutional neural network? why?
user avatar
  • 431
0 votes
1 answer
22 views

Textbook definition of Data augmentation

I'm trying to find a textbook definition of Data augmentation. We all know what it is, but I'm having a hard time finding a reliable source which I can cite for my paper: The Wikipedia article on ...
user avatar
  • 123
2 votes
1 answer
789 views

First perform data augmentation or normalization?

Should I first perform data augmentation or normalization in deep learning? I am mainly interested in 2D and 3D input data. In tutorials that I have seen so far the data augmentation always comes ...
user avatar
  • 141
1 vote
2 answers
268 views

Why should I use data augmentation as Keras layer

I've seen in several Tensorflow/Keras tutorials that data augmentation functions are added as keras layers. When I converted my Keras Python model (for production purpose) to TensorflowJS I faced the ...
user avatar
  • 113
3 votes
1 answer
1k views

What exactly are the data augmentation experimental Keras' layers doing?

From what I gathered, data augmentation consists in increasing your number of instances in your dataset by applying some transfromations. Let's say I want to classify images. If I apply a random ...
user avatar
  • 43
0 votes
1 answer
16 views

Are GPTs close to real intelligence or just another Data In -Data Out -Data Permutation and Combinations?

Use cases and Solutions surrounding GPT's have taken NLP world with storm and started the GPT-Best vs GPT- Not So Best war on the internet. There are solutions been derived from API's provided by HF. ...
user avatar
  • 339
1 vote
1 answer
57 views

Is there any possibility to apply deep dreaming in data augmentation?

I looked into the deep dreaming concepts and feel like this has the potential for data generation. But i'm not sure how possible this concept is. Any thoughts regarding this?
user avatar
  • 13
0 votes
0 answers
230 views

How to change voice features in python without affecting speech/language features?

I am trying to build a CNN model which should be able to identify the language being spoken in an audio file. I have extracted the MFCC matrix (for 13 coefficients) for each audio file and trained it....
user avatar
1 vote
2 answers
143 views

Problem of continuous training - Supervised learning

I am sure this is a most common problem, but would like to know by experts on how to tackle it. Note that, I mostly deal with textual data (NLP problems). When a supervised learning model is created, ...
user avatar
4 votes
2 answers
885 views

Why are results without Transfer Learning better than with Transfer Learning?

I developed a neural network for license plate recognition and used the EfficientNet architecture (https://keras.io/api/applications/efficientnet/#efficientnetb0-function) with and without pretrained ...
user avatar
  • 93
1 vote
0 answers
302 views

Data augmentation for recommendation systems

I have a user-item matrix that I use to train a denoising autoencoder to predict the top-k items to recommend to the different users. The idea is to corrupt the matrix by erasing a percentage ...
user avatar
  • 111
1 vote
1 answer
80 views

Training with less data

Most problem with machine learning projects I have faced is the lack of data. The samples available are enough to disqualify rule based approach but not enough for a neural network to train. For ...
user avatar
0 votes
2 answers
195 views

How to train a keras model on both original and augmented data from ImageDataGenerator?

I have a dataset that contains about 87000 images in a directory, with each class in a separate subfolder. I've tried the class ImageDataGenerator() and the ...
user avatar
1 vote
2 answers
428 views

Should augmentation also be performed on the validation set when the dataset is imbalanced?

I am training a CNN on images (2 classes) and I have an imbalanced dataset (1:7 ratio). I am trying to tackle this by performing offline image augmentation. Should I perform augmentation also on the ...
user avatar