Questions tagged [data-augmentation]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
20 views

What was used before data augmentation (e.g. SMOTE) to train ML models on imbalanced data? Please provide citations

I am seriously curious on how imbalance data was treated in machine learning and statistical learning before modern time data augmentation solutions such as SMOTE appeared. Please provide citations ...
Full Array's user avatar
0 votes
0 answers
10 views

Highly unbalnced text data giving very low matrics

I have an unbalanced multi-class banking text data with around 76 classes. Classes are badly distributed such as one class which is combination of 240 other different categories, represents 50% of ...
Remrem's user avatar
  • 1
0 votes
0 answers
18 views

Image classification of centered objects with convolutional neural networks

Given that I have a set of images that contain multiple objects for which labels exist and the object the image label refers to is always in the center. The objects vary in size. I want to train a ...
fhllw's user avatar
  • 1
0 votes
1 answer
63 views

Label_fields are not valid error when using Albumentations

I'm not sure if you can have duplicates cross-forums, but my previous question on Stack Overflow was never answered. I'll paste it here just in case. I'm using albumentations with the following code: <...
aSquaredRush's user avatar
0 votes
0 answers
143 views

Augmentation and transformations in Detectron2

I'm working on a custom Faster RCNN with Detectron2 framework and I have a doubt about transformation during training and inference. I created a custom Trainer ...
Agostino Dorano's user avatar
0 votes
1 answer
21 views

Investigating the Impact of Additive Gaussian Noise on EEG Signal Classification: Analyzing the Relationship between Augmented and Original Data

Definition: I have conducted research on EEG signal classification, specifically focusing on distinguishing between two different classes using raw EEG signals. Data availability poses a significant ...
Armin Amini's user avatar
0 votes
0 answers
19 views

what the best data augmentation for the time series data

I have force plate data and smartsole data. I want to make a regression model to predict force plate data using smartInsole data. I want to add variations to input and output data and find the ...
stack offer's user avatar
0 votes
1 answer
166 views

Does Rotation and Translation range in augmentation? (image data)

I am building a classification model. I want to use augmentation (less images + but no class imbalance) I want to use rotation and translation. Does it matter what range I use and how big the range is?...
Academic's user avatar
  • 482
0 votes
1 answer
43 views

How to perform a classification experiment with data augmentation?

I'm working on an audio classification experiment. In my original database, I have 1,412 records. To improve the performance of my models, I resorted to data augmentation, applying simple techniques ...
Paulo Sergio Moreira's user avatar
0 votes
0 answers
23 views

Data augmentation layer based on physical model for time series data

I am quite new to the Keras API, so forgive me if I use incorrect terminology and for my lack of knowledge about the API. This is for a mathematical (wave modelling) research project and I am quite ...
LightninBolt74's user avatar
0 votes
0 answers
23 views

GAN architectures for handling big size images

I am trying to generate new images in order to improve the number of samples in a dataset. I have noticed that the majority of the GANs are trained with quite small images. For example, ...
Simone's user avatar
  • 695
1 vote
0 answers
15 views

Which algorithm should I use for Data Augmentation of Time Sereis Data? [closed]

Can someone please recommend which Oversampling Algorithm technique should I use for time-series data set? Please share any sources or code to which I can refer. Thank You!
Kiran Maharana's user avatar
0 votes
0 answers
83 views

Why validation loss curve not decreasing of my ResNet50 model while training loss decresing? How I can improve without using transfer learning?

I am trying to understand my model by diagnosing the learning curve and how I can improve it. I aim to implement a deep-learning architecture (ResNet50) using a small dataset for binary classification ...
Mohammed Nurul Islam's user avatar
1 vote
0 answers
30 views

NLP - F1 score for positive class drops to 0 after data augmentation

I'm working on a 3-class text classification problem where my initial class distribution looked like this: positive: 50% negative: 25% and neutral: 25% And training on a model on this slightly ...
AnonymousMe's user avatar
0 votes
0 answers
29 views

What is the best way to acess the data in a pandas Window object?

I have a pandas DataFrame object, and I call its rolling method to get a Window object. Now I want to modify the data in each of the windows to get a new set of windows. Reading the source code, I ...
Bruno De Oliveira Macedo's user avatar
0 votes
0 answers
17 views

Is it correct to generate similar rows by reducing the time-frame of an instance?

I'm participating in a People Analytics project with a small historic dataset that includes event variables. The aim is to predict employee's attrition. I have variables like area, dept., company, etc....
ru.mp's user avatar
  • 63
0 votes
1 answer
222 views

Model does not learn when using Keras 'flow_from_directory', but learns fine with 'image_dataset_from_directory'?

When classifying images with Keras, I am able to achieve a validation accuracy around 90-95%, however, I am trying to improve with the use of augmentation so have switched from ...
BDI's user avatar
  • 1
1 vote
1 answer
267 views

CNN good results on train and test, bad results on real world data

I'm trying to build a neural network for an age detection task. Here some details : Dataset: I am using the "facial age" Kaggle dataset and the "UTKFace" dataset for a total of ...
Daniel_Fortesque's user avatar
0 votes
0 answers
39 views

Data Augmentation avoid RAM problem

I have a dataset (imgs) which is a list of numpy.ndarray: images of dimension (200, 200, 3). When I try to augment it with imgaug library I always occur in RAM ...
Daniel_Fortesque's user avatar
0 votes
0 answers
140 views

Replace a lookup table with machine learning

I have a lookup table with 2 input columns and 2 output columns. I want to replace it with a value function such that with a given input pair, the function can give the output pair with minimal error. ...
Tanim's user avatar
  • 1
1 vote
0 answers
23 views

What are some common data augmentation techniques used for code?

I understand that there are data augmentation techniques for natural language such as word/sentence shuffling, word replacement with synonyms and syntax-tree manipulation. However, I have hard time ...
jjwest's user avatar
  • 11
1 vote
1 answer
131 views

Certain Image Augmentation Prevent Unet Model from Learning

I am training a Unet model for cell image segmentation from microscopy images. In order to help the model generalize better to different microscopes, I attempted to apply brightness augmentation to ...
Berk Yalcinkaya's user avatar
1 vote
1 answer
122 views

How to do Data Augmentation efficiently in Tensorflow 2?

First of all I'm asking that because of this tutorial. When I heard about Data Augmentation the definition I learned was something like: "It's a technique where we create more data to our dataset ...
Tiago Martins's user avatar
0 votes
0 answers
20 views

Do we know why GAN-based data augmentation works?

Although I've seen many examples of GAN-generated synthetic data greatly improving the performance of models, I struggle to understand how this is possible. Say we are training a classifier $h$ to ...
Isaac Wasserman's user avatar
0 votes
1 answer
78 views

Data Augmentation Keras length of data

I'm confused when I add data augmentation should I get more data or the same data I tested my x_train length to confirm but I got the same length before augmentation and after augmentation is that ...
Okba's user avatar
  • 1
1 vote
0 answers
118 views

Can data augmentation techniques be misleading?

In an attempt to handle imbalance in data, especially in the case of extremely imbalanced data, can the various data augmentation techniques create some bias?
Deepak's user avatar
  • 53
0 votes
0 answers
145 views

Is There Techniques for creating synthetic Data for Regression Problem i tried SMOTE and its variant but these are for classification problem

This is my data "Volume" is my Target variable and all other are Independent variables i just applied labelencoder on Area_categ , wind_direction_labelencod and on current _label_encode and ...
elya abbas's user avatar
2 votes
1 answer
93 views

Data augmentation in images

Suppose there is a ML network that takes grayscale images as the input. The images that I have are RGB images. So, instead of converting these RGB images to grayscale, I treat each individual colour ...
user7080's user avatar
1 vote
1 answer
34 views

Does synthetic data be over sampled as well?

I'm building a binary text classifier, the ratio between the positives and negatives is 1:100 (100 / 10000). By using back translation as an augmentation, I was able to get 400 more positives. Then I ...
guestmember123456790's user avatar
1 vote
1 answer
314 views

Why we call Mix-up method is a data augmentation technique?

I am bit confused in the Mixup data augmentation technique, let me explain the problem briefly: What is Mixup For further detail you may refer to original paper . We double or quadruple the data ...
Ahmad's user avatar
  • 138
0 votes
0 answers
27 views

Augmentation for sound recognition of dog barks for CNNs

I am training CNNs to recognize dog barking, and for this I would like to augment the data sets I have (~30'000 10s clips with either barks, or no-barks in them). The straight forward idea was to mix ...
JumboJetlin's user avatar
1 vote
1 answer
349 views

ValueError: Error when checking input: expected time_distributed_6_input to have 5 dimensions, but got array with shape (32, 224, 224, 3) [closed]

I am trying to apply data augmentation to avoid overffiting in my CNN-LSTM image classification model. My training data has the shape: (1882, 1, 224, 224, 3) My ...
Baya Lina's user avatar
  • 111
1 vote
1 answer
353 views

how does zoom out works in data augmentation?

how does zoom out works in data augmentation? I'm reading a doc on data augmentation in Keras ,and it says that randomZoom(0.2) zooms in and out by a factor in rage ...
amir shakiba's user avatar
0 votes
2 answers
581 views

Removing outliers from a multi-dimensional dataset & Data augmentation

Removing the outliers of a single-dimensional data can be easily done by removing the points that are outside of the IQR range. But how should the process of detecting and removing outliers be done if ...
Centauri_42's user avatar
1 vote
0 answers
133 views

Methods to combine datasets from different time periods

Consider a multivariate time series forecasting task where I have two datasets A and B. A goes from 1960 to 2020 and B goes from 2010 to 2020. There is a feature f ...
kyc12's user avatar
  • 155
0 votes
2 answers
373 views

How much data augmentation is required on an imbalanced dataset?

Imagine I have a dataset with positive and negative sentences, and I need to train a transformer (Like BERT) to do the binary classification. The problem is that there are 100 negative sentences and ...
SMMousaviSP's user avatar
0 votes
1 answer
65 views

Batch normalization for multiple datasets?

I am working on a task of generating synthetic data to help the training of my model. This means that the training is performed on synthetic + real data, and tested on real data. I was told that batch ...
Manveru's user avatar
  • 111
1 vote
0 answers
68 views

Specify torchvision transforms depending on the properties of an image and a mask

I have a dataset 1000 of images and corresponding segmentation masks from dermatologists. The images come in different sizes (as low as 400x600 and as large as 4Kx4K). 95% of image pixels are not ...
sixtytrees's user avatar
0 votes
0 answers
39 views

How can I get dark, median-brightness and bright distribution histograms from an image?

I'm trying to replicate an augmentation technique used in this paper. This is how they explain the procedure: [...] the augmentation technique was used to adjust the histogram because the intensity ...
Juan Pablo S. G.'s user avatar
0 votes
1 answer
288 views

What is the scope of Keras' ImageDataGenerator.flow_from_dataframe seed parameter?

I've been working on a U-Net model using training images stored on my local drive. To load these I have been using Keras' ImageDataGenerator.flow_from_dataframe ...
Ali's user avatar
  • 111
1 vote
0 answers
18 views

Working on an image classification project (microscopic images) , have some doubts [closed]

Currently, I am working on an image classification project. The data set contains very high resolution images taken via an electron microscope. Hence, I have few and limited instances. I have done EDA ...
Aditi's user avatar
  • 11
5 votes
1 answer
4k views

Why does adding data augmentation decrease training accuracy a tiny bit?

Before data augmentation, my model clearly overfits and hits a 100% training accuracy and a 52% validation accuracy. When only adding data augmentation with Keras, as a regularization technique, it ...
Kralley's user avatar
  • 53
0 votes
1 answer
428 views

Does Mixup requires two loss functions?

I created a neural network with multi-label classification using MSE. Now, I would like to use Mixup. Do I need two loss functions (for each target one) or is the result the same if I just combine the ...
codeprof's user avatar
1 vote
1 answer
547 views

How to mantain the nested structure of a tf.dataset after applying map?

I'm creating a tf.dataset object containing 2 images as inputs and a mask as target. All of them are 3D in grayscale. After applying a custom map, the shape of the object changes from ...
fgarciador's user avatar
0 votes
1 answer
449 views

How to implement random cropping during training?

I'm developing a U-net like model which segments the damaged tissue of the brain between two time-points in Multiple Sclerosis patients. The model is given the baseline and follow-up images as x and ...
fgarciador's user avatar
1 vote
1 answer
3k views

Rescale parameter in data augmentation

I'm a little bit lost about the rescale parameter in the ImageDatagenerator function. I know that the rescale argument by itself does not augment my data and that ...
Bia's user avatar
  • 113
0 votes
1 answer
2k views

How many images are generated when ImageDataGenerator is used, and when data augmentation is included as a part of the model?

Is there any way to know the number of images generated by the ImageDataGenerator class and loading data using flow_from_directory method? I searched everywhere for the same but couldn't find anything ...
Hrushikar Teja K's user avatar
3 votes
0 answers
341 views

Is it possible to increase the number of images of one class using data augmentation, which is not applied on the other class, in the same dataset?

I have 2 classes for my image classification problem, say class A and class B, and I am using tensorflow and keras for the same. One of them have around 5K images while the other have just around 2K ...
Hrushikar Teja K's user avatar
2 votes
2 answers
735 views

What is the use of using width/height shift in data augmentation?

I'm not sure to understand the use of augmentation data using width shift and height shift. Say I have limited image data, and I want to create new data using Keras' ImageDataGenerator. To classify ...
JPeterson's user avatar
0 votes
0 answers
24 views

Should I resize my object localization dataset in order to train a fully convolutional network?

I want to create a fully-convolutional neural net that trains on wider face datasets in order to draw bounding box around faces. The dataset is highly diverse in the image sizes. So my question is, ...
Xanta_Kross's user avatar