Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

Filter by
Sorted by
Tagged with
1 vote
2 answers
191 views

Is there a point in hyperparameter tuning for Random Forests?

I have a binary classification task with substantial class imbalance (99% negative - 1% positive). I want to developed a Random Forest model to make prediction, and after establishing a baseline (with ...
1 vote
2 answers
776 views

dataset split for image classification

I am trying to do image classification for 14 categories (around 1000 images for each cat). And i initially created two folders for training and validation. In this case, do I still need to set a ...
3 votes
2 answers
5k views

Minimum number of samples to train XGBoost without overfitting

When using Neural Networks for image processing I learned a rule of thumb: to avoid overfitting, supply at least 10 training examples for every neuron. Is there a similar rule of thumb for classifiers ...
2 votes
1 answer
110 views

When to stop the final model training?

Let's say I'm participating in a Kaggle image recognition competition. Firstly, I create a train/validation split and find the good hyperparameters for my model. Here the stopping criterion is when ...
1 vote
1 answer
85 views

Why is it so common to focus only validation performance during hyper-parameter optimization

Assuming a standard train/validation/test split, the common practice is (a) to train multiple models with different hyper-parameter configurations on the training set, (b) to evaluate these models ...
0 votes
2 answers
238 views

How to train a keras model on both original and augmented data from ImageDataGenerator?

I have a dataset that contains about 87000 images in a directory, with each class in a separate subfolder. I've tried the class ImageDataGenerator() and the ...
0 votes
1 answer
72 views

why my accuracy and recall become higher in testing than training. How to interpret

Why in hidden layer 2 and 3 in neural networks scratch, the accuracy and recall I got low, but in testing the accuracy and recall become higher. In hidden layer 4 it's get weird when sampling strategy ...
0 votes
1 answer
80 views

Overfitting still exists using different techniques on voice classification

I have 986 voice signals which have been collected by our team. The data set includes 745 healthy and 150 unhealthy voice signals. I split the data into 70% training and 20% validation and 10% test (...
1 vote
1 answer
270 views

Bounding box regression without a classification task

I am using PyTorch to create a model that detects certain objects in an image. I framed my problem as a regression on bounding boxes, without any classification task whatsoever. The reasoning behind ...
1 vote
2 answers
189 views

Training Object Detection model on just 10 images

I am trying to train an object detection model using Mask-RCNN with Resnet50 as backbone. I am using the pre-trained models from PyTorch's Torchvision library. I have only 10 images that I can use to ...
0 votes
0 answers
98 views

How to recognize if a model is overfitting?

I'm trying to develop a real-time YOLOv8 model for detecting falls in a home environment. The dataset I used consists of approximately 1100 images labeled as "fall" and "nofall," ...
0 votes
1 answer
71 views

How do I know If my regression model is underfitting?

How do we evaluate the performance of a regression model with a certain RMSE given that a domain knowledge performance metric is not present? Maybe MAPE is one way of comparing the performance of my ...
0 votes
2 answers
2k views

How to perform Multi-Label Image Classification with EfficientNet

Problem My goal is to perform multi-label image classification with EfficientNet. It should take a picture as input and e.g. tell the user that it sees a person AND a dog on the picture, meaning the ...
3 votes
3 answers
176 views

Overfitted model produces similar AUC on test set, so which model do I go with?

I was trying to compare the effect of running GridSearchCV on a dataset which was oversampled prior and oversampled after the training folds are selected. The oversampling approach I used was random ...
2 votes
1 answer
320 views

Multilabel Classification - Overfitting?

My task is the following: To input drug combinations and output renal failure-related symptoms from the drug combinations. Both the drug combinations and renal-failure related symptoms are represented ...
0 votes
2 answers
488 views

How many features do I select when doing feature selection for regression algorithms? Is R2 and RMSE good measures of success for overfitting?

Context: I'm currently crafting and comparing machine learning models to predict housing data. I have around 32000 data points, 42 features, and I'm predicting housing price. I'm comparing Random ...
0 votes
1 answer
85 views

Difference between Validation Error on Learning Curve and Validation Error Calculation in Machine Learning Model

I am encountering a problem where the validation error I see on the learning curve of my machine learning model is different from the validation error I calculate using the mean squared error function....
1 vote
1 answer
3k views

Overfitting in Huggingface's TFBertForSequenceClassification

I'm using Huggingface's TFBertForSequenceClassification for multilabel tweets classification. During training the model archives good accuracy, but the validation accuracy is poor. I've tried to solve ...
0 votes
1 answer
45 views

Why is my genetic algorithm overfitting so much?

I'm only training on a fraction of the data each generation: ...
2 votes
1 answer
1k views

Overfitting in K-means

How do you test your results for overfitting in a k-means run? Some people have said use a training set. I have about 1500 records and about 20 fields.
0 votes
1 answer
57 views

Is my model overfitting based on my accuracy/loss curves?

Do those results indicate that my model is overfitting?
0 votes
1 answer
46 views

VGG16 Transfer Learning for image binary classification - suspected overfitting

I'm using VGG16 for transfer learning on a binary image classification task about human posture. The sample totaled about 2,000 images, with about 900 and 1,000 images in each category, respectively. ...
1 vote
1 answer
254 views

How to curve fit, Z variable dependent on X and Y?

I'm trying to find the function for this visualization: I would like to get feedback if I'm taking the right approach. My approach: These data points are created by a person. They are two ...
2 votes
1 answer
125 views

How to analyze neural network quality in case of overfitting?

I have a Keras neural network that has images both as input and reference data. My network demonstrates overfitting (for example, train accuracy is about 80% but test accuracy is only up to 70%) due ...
1 vote
1 answer
830 views

How do I know if this model is overfitting?

This is my example R script for a decision tree: ...
2 votes
2 answers
654 views

CNN + LSTM model for images performs poorly on validation data set

My training and loss curves look like below and yes, similar graphs have received comments like "Classic overfitting" and I get it. My model looks like below, ...
0 votes
1 answer
230 views

Learning curves

I am working on a multiclass classification problem. I want to know whether my model is overfitting or underfitting. I am learning how to plot learning curves and have 4 doubts. 1.) Is the ordering of ...
0 votes
1 answer
89 views

Random Forest overfitting to unbalanced data set

I am working on an unbalanced classification problem. I have have 2000 points which are positive, and 6000 points as -ve (chosen randomly from 100k universe of -ve points universe). Although I have ~...
1 vote
1 answer
577 views

Will repeatedly fine-tuning on new data cause overfitting?

I have a binary classification model which I have trained on a training set. On the validation set its accuracy is ~85%. I set up early stopping which ended training when validation loss increased. ...
0 votes
1 answer
673 views

Precision, recall and accuracy metrics significantly different between training/validation and actual predictions

I have two sequential models built with Keras that train on data from a CSV file. This is how they are built ...
1 vote
1 answer
258 views

Is there any concern for a pretrained model to overfitting to a fine-tuning task that has overlapping pretraining and training data?

Let's say my language model is pretrained on a general text corpus, and I want to use it for some specific downstream task that has it's datasets also included in the general corpus, is there any ...
0 votes
0 answers
74 views

Can't overfit Transformer Encoder

In the below code I am trying to train a very simple Transformer Encoder model to basically do nothing with its input. Giving some arbitrary input vector x, the aim of the model is then to output that ...
0 votes
1 answer
673 views

Number of units for first layer in Keras Sequential Model

I have a huge CSV structured dataset. I'm feeding that dataset to a Keras Sequential Model. My question is, can my Model have number of units greater than the number of input features? At the moment, ...
0 votes
1 answer
326 views

What causes explosion in MSE when training?

I (probably) well overfitted/overtrained a model. But I was just curious as to what might cause this type of behaviour. I carried on training (Epoch 1/50 is not the first epoch of training this model)....
0 votes
0 answers
23 views

How to reduce the overfitting in my CNN model?

I am new in this world want practice for create a convolutional neuronal network. A model convolutional for image classification. I want classificate women and men images. Previous, I did a course by ...
0 votes
0 answers
19 views

overfitting or not

Hello so i'm building a classification model i train my on various models and these are the metrices so i want to know if ther's an overfitting or not
0 votes
2 answers
120 views

Is it possible to overfit a simple single variable linear regression model?

I searched this question and the answer I got was about a general regression model, rather than a single variable, linear regression model. If you increase the number of variables, you could fit a ...
0 votes
1 answer
187 views

Amount of data needed for deep learning vs support vector machine

I often read about the fact, that the amount of data to train and get a generalizing model for a deep learning algorithm is much higher in comparison, e.g. to a support vector machine. It makes sense, ...
0 votes
1 answer
195 views

Training ResNet50 model for binary classification

I want to use ResNet50 model to perform binary classification on a dataset spectrogram dataset. In order to do that I had to make a couple of modifications to the model's architecture: Modified the ...
1 vote
2 answers
107 views

Xgboost model predicting extreme values for events and non-events | Overfitting

Extreme values are predicted by my trained xgboost classification model in BQML for both events (Y=1) and non-events (Y=0). For all event observations, the model calculates probability scores that ...
0 votes
0 answers
28 views

Using Embedding For Regularization

Is using embeddings for regularization a valid practice? My reasoning for that is that encoding training/tests datasets into smaller vectors would allow a smaller network with fewer parameters and ...
0 votes
1 answer
24 views

Challenges in Predicting Molecule Activity

I want to share a concern I have. I want to obtain a machine learning model that can predict whether a molecule exhibits biological activity. For this purpose, I have a set of molecules that do ...
1 vote
0 answers
17 views

Avoid overfitting to noise by a noise penalty approach instead of early stopping?

I came across this article on deep learning for computational MRI and found an interesting sentence "However, early stopping has to be performed to not overfit to the noisy measurements." ...
0 votes
0 answers
59 views

Classification Threshold Optimization after GridSearchCV

In my machine learning problem I am using a CNN to classify images. Since my dataset is imbalanced I want to perform classification probability threshold tuning so I can find the optimal balance ...
0 votes
0 answers
12 views

Test accuracy plateaus when increasing max_depth -> inf

I've built a Random Forest model that classifies into four categories based on around 10 input features. To test the accuracy, I performed 5-fold stratified cross validation using the ...
0 votes
0 answers
28 views

I defined a subclass of torch.utils.data.IterableDataset, which line of the following loader could have caused the issue and why?

The current model is overfitting quickly, i.e., the training error is minimized in a few epochs while the validation error remains high. Suppose training data is sufficient, i.e. self.cohort is set ...
0 votes
1 answer
86 views

How to compare test vs train model performance

When comparing the test vs train model performance to ensure no overfitting (e.g., using AUC ROC as an example), is it better to select the model with the largest test score, or the model with the ...
0 votes
0 answers
49 views

Learning curve - Why does the train learning curve is flat?

I implemented a model in which I use Random Forest as classifier and I wanted to plot the learning curves for both training and test sets to decide what to do next in order to improve my model. ...
0 votes
0 answers
47 views

The cost function gets stuck at 120 epochs

I did a neural network in c++ to recognize handwritten digits using the MNIST dataset without any neural network pre-existing libraries. My network has 784 inputs neuron (the pixel of the image), 100 ...
0 votes
1 answer
32 views

model.evaluate gives low results?

i have an image dataset and there are 6300 images with 5 classes . The features extracted and dataset reduced to 256 features. This dataset gives good results(%99) when tested ANN with Backpropagation(...

1
2 3 4 5
8