Skip to main content
17 votes
Accepted

When using Data augmentation is it ok to validate only with the original images?

You should validate only on the original images. The augmentation is there so that it can help your model generalize better, but to evaluate your model you need actual images, not transformed ones. ...
Djib2011's user avatar
  • 7,998
13 votes
Accepted

CNN - imbalanced classes, class weights vs data augmentation

Is this approach better than the mere augmentation or just the use of class weights ? Note that data augmentation is the process of changing the training samples (e.g. for images, flipping them, ...
Esmailian's user avatar
  • 9,352
10 votes

When using Data augmentation is it ok to validate only with the original images?

Ideally, data augmentation is a step in your training pipeline, which comes after splitting your data into train/validation/test sets. Otherwise, you have the same data point in both training and ...
Bruno Lubascher's user avatar
7 votes
Accepted

How can I augment my image data?

You begin by asking about image normalisation, but then refer to other techniques, which I believe all fall under "image augmentation". So I will answer the more general question: how can I perform ...
n1k31t4's user avatar
  • 14.9k
5 votes
Accepted

Data Augmentation recommended pipeline

Out of the two pipelines you mentioned, I'd recommend the second (i.e. real-time augmentation). This is better than the first, because by performing random augmentations the network sees different ...
Djib2011's user avatar
  • 7,998
4 votes

How can I augment my image data?

Disclaimer: I will try to answer the question but promote Image Augmentation Library Albumentations, which may collaborators and I develop in free time and which we believe is the best image ...
Vladimir's user avatar
  • 141
4 votes
Accepted

First perform data augmentation or normalization?

In the specific context of image/video problems, it may be okay to normalize the data before augmentation, because you already know that each feature (pixel) will have a value between 0 and 255. As ...
zachdj's user avatar
  • 2,734
3 votes
Accepted

Why can't I use data augmentation with a pretrained convnet?

Found the answer in stats.stackexchange.com. Hopefully this helps anyone else with the same question. feature extraction: freezing convolutional base vs. training on extracted features
nvi's user avatar
  • 69
3 votes
Accepted

Is there a disadvantage to letting a model train for a large number of epochs?

If you can be sure that the model is not seeing the same instances repeatedly then there is very good chances that your model is not overfitting and that is precisely what you can measure from your ...
JahKnows's user avatar
  • 8,946
3 votes
Accepted

Are there libraries or techniques for 'noisifying' text data?

I you want some kind of data-sets like Google spell checking data I suggest you look into the The WikEd Error Corpus dataset. The corpus consists of more than 12 million sentences with a total of 14 ...
Dani Mesejo's user avatar
  • 2,226
3 votes

Data Augmentation for Regression

Yes, you can perturb your data (and targets) in ways that you wish your model to be robust against, for example by adding small amounts of noise (possibly Gaussian) or synthetic anomalies, or by ...
Michael Brundage's user avatar
3 votes

Images Dataset Augmentation with fixed parameters of Crop & Rotation Angle

If you need to work on images using Python, the preferred library is PIL. Here I show a function to do the modifications you have delimited. This code makes no effort to manage multiple files, or ...
Stephen Rauch's user avatar
  • 1,783
3 votes

Data Augmentation in videos

You can augment videos in the temporal dimension through clipping, or taking random sequences of consecutive frames. You can also augment in the spatial dimension by cropping frames randomly to ...
liangjy's user avatar
  • 391
3 votes
Accepted

Why are results without Transfer Learning better than with Transfer Learning?

As @fuwiak mentioned, transfer learning may not work if pre-trained model has been fitted on a "very different" dataset. Typically if the pre-trained network extract information that is not ...
etiennedm's user avatar
  • 1,405
3 votes
Accepted

Why does adding data augmentation decrease training accuracy a tiny bit?

The obvious reasons why data augmentation might reduce the train accuracy is - As you know, deep learning models are data hungry. If the model don't get enough data to recognize the patterns then it ...
Devashish Prasad's user avatar
3 votes
Accepted

How to do Data Augmentation efficiently in Tensorflow 2?

For most frameworks, random augmentation includes no augmentation (random flipping may either flip or not, random rotation angle can be 0 or nigh). This is also randomized every epoch (or whenever ...
dx2-66's user avatar
  • 696
2 votes
Accepted

Why augmenting the training data with binary attributes works better for our dataset?

Your third model has the most capacity of the three. When you have the values $x$ with special negative value $\xi$ and indicators $\mathbf{1}_0$ and $\mathbf{1}_{\xi}$, the linear model can fit $$ \...
Ben Reiniger's user avatar
  • 11.9k
2 votes

Is data augmentation changing the train/test sets distribution?

I would argue that augmentation does not alter your train/test sets distribution. I say this because I consider data augmentation to be part of your training ...
Bruno Lubascher's user avatar
2 votes

How to implement PCA color augmentation as discussed in AlexNet

You should not apply *255. delta was supposed to be added to renorm_image, because you ...
user10253771's user avatar
2 votes
Accepted

Generated training set on convnet

I don't think it is wise. Your intention to do validation on your real data is correct. But the way you have it now your model will be prevented from training on data that is from the same ...
Simon Larsson's user avatar
2 votes

How to benefit Data augmentation when it yields to different classes

You could rotate images manually (without using ImageDataGenerator) and save it to disk. That way you would know which images you have rotated - so you would know ...
Antonio Jurić's user avatar
2 votes

Is there any augmentation tool for images and bounding boxes?

Other widely used Python libraries for data augmentation include: OpenCV: has functions/methods for bounding boxes, changing color space, scaling, cropping, translation, rotation, filters, blur, ...
AlexK's user avatar
  • 350
2 votes
Accepted

Is there any augmentation tool for images and bounding boxes?

I think you should look into imgaug. It supports most image augmentation and does have support for bounding boxes. Docs: https://imgaug.readthedocs.io/en/latest/
Simon Larsson's user avatar
2 votes

Why do we use different image processing at train and test time for an image classification tasl

The key difference is that for training the preprocessing serves the purpose of data augmentation: it increases variance in the training data and thereby helps to better generalize. That is why you ...
Jonathan's user avatar
  • 5,430
2 votes
Accepted

Data augmentation for multiple output heads in Keras

Please refer to the source code provided at https://gist.github.com/swghosh/f728fbba5a26af93a5f58a6db979e33e which should assist you in writing custom generators (basis ImageDataGenerator) for ...
Swarup Ghosh's user avatar
2 votes

How to do data augmentation for Machine Learning on statistical data?

You can do mainly two things: bootstrap or oversampling. With statistical data you can do bootstrapping (random sampling with replacement) Bagging methods help boosting you model accuracy. The ...
Carlos Mougan's user avatar
2 votes

Training with less data

As rightly pointed out by @erwan, it is a bad idea to use data augmentation with 'text data' The problem of 'training with less data' can be approached in many ways, here I enlist two ways which ...
YoungSheldon's user avatar
2 votes

How many images are generated when ImageDataGenerator is used, and when data augmentation is included as a part of the model?

In one epoch - It's the number of images in your Directory or the DataFrame In case of a custom Generator. It will be batch_size * steps_per_epoch You may check ...
10xAI's user avatar
  • 5,634
2 votes
Accepted

Rescale parameter in data augmentation

As rightly pointed out by you the rescale=1./255 will convert the pixels in range [0,255] to range [0,1]. This process is also called Normalizing the input. Scaling ...
prashant0598's user avatar
  • 1,511
2 votes
Accepted

Data augmentation technique not working correctly

You should measure the performance on the same dataset to be a good head-to-head comparison. I suggest you create a non-augmented test set first, then train a simple SDG from the train set, and then ...
saiRegrefree's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible