Questions tagged [data-augmentation]
The data-augmentation tag has no usage guidance.
114 questions
0
votes
0
answers
24
views
Question about the limitations of regularization
I am training a neural network which is overfitting. Even when I increase the number of parameters, the test lost plateaus while the training loss keeps decreasing. Can regularization (like an L1 or ...
0
votes
0
answers
51
views
While working on binary image classification, the class mode set to binary incorrectly labels the images, but does it correct on categorical
I am currently working on a binary image classification. My problem is that when i use data augmentation, it incorrectly labels the images when it is set to binary.
The things i have tried:
Looked ...
0
votes
1
answer
18
views
Data augmentation technique not working correctly
Write a function that can shift an MNIST image in any direction (left, right, up, or down) by one pixel.6 Then, for each image in the training set, create four shifted copies (one per direction) and ...
- 115
1
vote
0
answers
60
views
How to solve imbalanced dataset oversampling problem in multi labels-classes instance segmentation task?
I want to use models YOLOv7-seg for instance segmentation of tree species in images. There are 26 species of trees, and each image may contain multiple species. There is a distinction between dominant ...
- 11
1
vote
0
answers
136
views
Generating quality synthetic tabular data - is it possible when one's dataset is extremely small?
I've got a dataset consisting of only 17 samples and 6 continuous features (all values in the dataset contain decimals, although 2 features exhibit categorical-ish behaviour). I'm looking at the ...
- 11
1
vote
2
answers
125
views
Is a Random Forest Capable of Learning and Predicting Numerical Trends in Panel Data?
In a panel data set consisting of exponential functions, each indexed by an integer i ranging from 0 to 100. The exponential function is defined as f(i, t) = A(i) * e^(-r(i) * t), where A(i) is the ...
- 151
0
votes
0
answers
26
views
What was used before data augmentation (e.g. SMOTE) to train ML models on imbalanced data? Please provide citations
I am seriously curious on how imbalance data was treated in machine learning and statistical learning before modern time data augmentation solutions such as SMOTE appeared.
Please provide citations ...
- 226
0
votes
1
answer
215
views
Label_fields are not valid error when using Albumentations
I'm not sure if you can have duplicates cross-forums, but my previous question on Stack Overflow was never answered. I'll paste it here just in case.
I'm using albumentations with the following code:
<...
0
votes
1
answer
58
views
Investigating the Impact of Additive Gaussian Noise on EEG Signal Classification: Analyzing the Relationship between Augmented and Original Data
Definition: I have conducted research on EEG signal classification, specifically focusing on distinguishing between two different classes using raw EEG signals. Data availability poses a significant ...
- 21
0
votes
1
answer
552
views
Does Rotation and Translation range in augmentation? (image data)
I am building a classification model. I want to use augmentation (less images + but no class imbalance)
I want to use rotation and translation. Does it matter what range I use and how big the range is?...
- 482
0
votes
1
answer
47
views
How to perform a classification experiment with data augmentation?
I'm working on an audio classification experiment. In my original database, I have 1,412 records. To improve the performance of my models, I resorted to data augmentation, applying simple techniques ...
0
votes
0
answers
27
views
Data augmentation layer based on physical model for time series data
I am quite new to the Keras API, so forgive me if I use incorrect terminology and for my lack of knowledge about the API.
This is for a mathematical (wave modelling) research project and I am quite ...
- 101
1
vote
0
answers
15
views
Which algorithm should I use for Data Augmentation of Time Sereis Data? [closed]
Can someone please recommend which Oversampling Algorithm technique should I use for time-series data set? Please share any sources or code to which I can refer. Thank You!
1
vote
0
answers
38
views
NLP - F1 score for positive class drops to 0 after data augmentation
I'm working on a 3-class text classification problem where my initial class distribution looked like this:
positive: 50%
negative: 25% and
neutral: 25%
And training on a model on this slightly ...
- 225
0
votes
1
answer
285
views
Model does not learn when using Keras 'flow_from_directory', but learns fine with 'image_dataset_from_directory'?
When classifying images with Keras, I am able to achieve a validation accuracy around 90-95%, however, I am trying to improve with the use of augmentation so have switched from ...
- 1
1
vote
1
answer
587
views
CNN good results on train and test, bad results on real world data
I'm trying to build a neural network for an age detection task.
Here some details :
Dataset: I am using the "facial age" Kaggle dataset and the "UTKFace" dataset for a total of ...
1
vote
0
answers
29
views
What are some common data augmentation techniques used for code?
I understand that there are data augmentation techniques for natural language such as word/sentence shuffling, word replacement with synonyms and syntax-tree manipulation.
However, I have hard time ...
- 11
1
vote
1
answer
168
views
Certain Image Augmentation Prevent Unet Model from Learning
I am training a Unet model for cell image segmentation from microscopy images. In order to help the model generalize better to different microscopes, I attempted to apply brightness augmentation to ...
1
vote
1
answer
169
views
How to do Data Augmentation efficiently in Tensorflow 2?
First of all I'm asking that because of this tutorial.
When I heard about Data Augmentation the definition I learned was something like: "It's a technique where we create more data to our dataset ...
- 158
0
votes
1
answer
102
views
Data Augmentation Keras length of data
I'm confused when I add data augmentation should I get more data or the same data I tested my x_train length to confirm but I got the same length before augmentation and after augmentation is that ...
- 1
1
vote
0
answers
129
views
Can data augmentation techniques be misleading?
In an attempt to handle imbalance in data, especially in the case of extremely imbalanced data, can the various data augmentation techniques create some bias?
- 53
0
votes
0
answers
192
views
Is There Techniques for creating synthetic Data for Regression Problem i tried SMOTE and its variant but these are for classification problem
This is my data "Volume" is my Target variable and all other are Independent variables i just applied labelencoder on Area_categ , wind_direction_labelencod and on current _label_encode and ...
2
votes
1
answer
123
views
Data augmentation in images
Suppose there is a ML network that takes grayscale images as the input. The images that I have are RGB images. So, instead of converting these RGB images to grayscale, I treat each individual colour ...
- 33
1
vote
1
answer
48
views
Does synthetic data be over sampled as well?
I'm building a binary text classifier, the ratio between the positives and negatives is 1:100 (100 / 10000).
By using back translation as an augmentation, I was able to get 400 more positives.
Then I ...
1
vote
1
answer
486
views
Why we call Mix-up method is a data augmentation technique?
I am bit confused in the Mixup data augmentation technique, let me explain the problem briefly:
What is Mixup
For further detail you may refer to original paper .
We double or quadruple the data ...
- 158
0
votes
0
answers
31
views
Augmentation for sound recognition of dog barks for CNNs
I am training CNNs to recognize dog barking, and for this I would like to augment the data sets I have (~30'000 10s clips with either barks, or no-barks in them).
The straight forward idea was to mix ...
1
vote
1
answer
486
views
ValueError: Error when checking input: expected time_distributed_6_input to have 5 dimensions, but got array with shape (32, 224, 224, 3) [closed]
I am trying to apply data augmentation to avoid overffiting in my CNN-LSTM image classification model. My training data has the shape:
(1882, 1, 224, 224, 3)
My ...
- 111
1
vote
1
answer
418
views
how does zoom out works in data augmentation?
how does zoom out works in data augmentation?
I'm reading a doc on data augmentation in Keras ,and it says that randomZoom(0.2) zooms in and out by a factor in rage ...
- 21
0
votes
2
answers
972
views
Removing outliers from a multi-dimensional dataset & Data augmentation
Removing the outliers of a single-dimensional data can be easily done by removing the points that are outside of the IQR range. But how should the process of detecting and removing outliers be done if ...
- 13
1
vote
0
answers
183
views
Methods to combine datasets from different time periods
Consider a multivariate time series forecasting task where I have two datasets A and B. A goes from 1960 to 2020 and B goes from 2010 to 2020. There is a feature f ...
- 165
0
votes
2
answers
540
views
How much data augmentation is required on an imbalanced dataset?
Imagine I have a dataset with positive and negative sentences, and I need to train a transformer (Like BERT) to do the binary classification. The problem is that there are 100 negative sentences and ...
- 101
0
votes
1
answer
92
views
Batch normalization for multiple datasets?
I am working on a task of generating synthetic data to help the training of my model. This means that the training is performed on synthetic + real data, and tested on real data.
I was told that batch ...
- 111
1
vote
0
answers
74
views
Specify torchvision transforms depending on the properties of an image and a mask
I have a dataset 1000 of images and corresponding segmentation masks from dermatologists. The images come in different sizes (as low as 400x600 and as large as 4Kx4K). 95% of image pixels are not ...
- 201
0
votes
1
answer
325
views
What is the scope of Keras' ImageDataGenerator.flow_from_dataframe seed parameter?
I've been working on a U-Net model using training images stored on my local drive. To load these I have been using Keras' ImageDataGenerator.flow_from_dataframe ...
- 111
1
vote
0
answers
19
views
Working on an image classification project (microscopic images) , have some doubts [closed]
Currently, I am working on an image classification project. The data set contains very high resolution images taken via an electron microscope. Hence, I have few and limited instances.
I have done EDA ...
- 11
6
votes
1
answer
6k
views
Why does adding data augmentation decrease training accuracy a tiny bit?
Before data augmentation, my model clearly overfits and hits a 100% training accuracy and a 52% validation accuracy. When only adding data augmentation with Keras, as a regularization technique, it ...
- 63
0
votes
2
answers
612
views
Does Mixup requires two loss functions?
I created a neural network with multi-label classification using MSE.
Now, I would like to use Mixup. Do I need two loss functions (for each target one) or is the result the same if I just combine the ...
- 1
1
vote
1
answer
692
views
How to mantain the nested structure of a tf.dataset after applying map?
I'm creating a tf.dataset object containing 2 images as inputs and a mask as target. All of them are 3D in grayscale. After applying a custom map, the shape of the object changes from ...
- 13
0
votes
1
answer
509
views
How to implement random cropping during training?
I'm developing a U-net like model which segments the damaged tissue of the brain between two time-points in Multiple Sclerosis patients. The model is given the baseline and follow-up images as x and ...
- 13
1
vote
1
answer
3k
views
Rescale parameter in data augmentation
I'm a little bit lost about the rescale parameter in the ImageDatagenerator function.
I know that the rescale argument by itself does not augment my data and that ...
- 113
0
votes
1
answer
2k
views
How many images are generated when ImageDataGenerator is used, and when data augmentation is included as a part of the model?
Is there any way to know the number of images generated by the ImageDataGenerator class and loading data using flow_from_directory method? I searched everywhere for the same but couldn't find anything ...
3
votes
0
answers
407
views
Is it possible to increase the number of images of one class using data augmentation, which is not applied on the other class, in the same dataset?
I have 2 classes for my image classification problem, say class A and class B, and I am using tensorflow and keras for the same. One of them have around 5K images while the other have just around 2K ...
2
votes
2
answers
830
views
What is the use of using width/height shift in data augmentation?
I'm not sure to understand the use of augmentation data using width shift and height shift.
Say I have limited image data, and I want to create new data using Keras' ImageDataGenerator. To classify ...
- 23
1
vote
1
answer
87
views
Is it better to augment both training and validation sets or just training set?
Is it better to augment data both training and validation sets or just the training set in order to achieve the best accuracy possible on a convolutional neural network? why?
- 461
0
votes
1
answer
31
views
Textbook definition of Data augmentation
I'm trying to find a textbook definition of Data augmentation. We all know what it is, but I'm having a hard time finding a reliable source which I can cite for my paper:
The Wikipedia article on ...
- 123
3
votes
1
answer
3k
views
First perform data augmentation or normalization?
Should I first perform data augmentation or normalization in deep learning? I am mainly interested in 2D and 3D input data. In tutorials that I have seen so far the data augmentation always comes ...
- 161
1
vote
2
answers
777
views
Why should I use data augmentation as Keras layer
I've seen in several Tensorflow/Keras tutorials that data augmentation functions are added as keras layers. When I converted my Keras Python model (for production purpose) to TensorflowJS I faced the ...
- 123
3
votes
1
answer
2k
views
What exactly are the data augmentation experimental Keras' layers doing?
From what I gathered, data augmentation consists in increasing your number of instances in your dataset by applying some transfromations. Let's say I want to classify images. If I apply a random ...
- 43
0
votes
1
answer
20
views
Are GPTs close to real intelligence or just another Data In -Data Out -Data Permutation and Combinations?
Use cases and Solutions surrounding GPT's have taken NLP world with storm and started the GPT-Best vs GPT- Not So Best war on the internet. There are solutions been derived from API's provided by HF. ...
- 369
1
vote
1
answer
108
views
Is there any possibility to apply deep dreaming in data augmentation?
I looked into the deep dreaming concepts and feel like this has the potential for data generation. But i'm not sure how possible this concept is. Any thoughts regarding this?
- 13