Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

66 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6
votes
0answers
68 views

XGBoost skews towards minority class

I have a dataset with 85k positive labels and 53k negative labels. For this use-case, I am trying to maximize my efforts to the negative class (accurately identify true negatives, and minimize false ...
3
votes
0answers
274 views

Knowing when a GAN is overfitting (sequence classification study)

I have sequences of long, sparse 1_D vectors (3000 digits, made of of 0s and 1s) that I am trying to classify. I have previously implemented a simple CNN to classify them with relative success (with ...
3
votes
1answer
201 views

Convolution neural network with 11 million parameters unable to overfit on 100 image samples

I have been trying to do some sort of image enhancement on grayscale images. I have used both pixel wise loss and perceptual loss (perceptual loss uses classifier between 2 classes trained on the same ...
3
votes
2answers
105 views

Overfitted model produces similar AUC on test set, so which model do I go with?

I was trying to compare the effect of running GridSearchCV on a dataset which was oversampled prior and oversampled after the training folds are selected. The oversampling approach I used was random ...
2
votes
1answer
29 views

machine learning disasters

I am writing a research paper and I am looking for reliable sources that provide information on disasters of machine learning. Especially in the field of autonomous driving. Have there been any ...
2
votes
1answer
35 views

Why Continous Variable Buckets Overfitting model

I have a continuous (high cardinal discrete) variable 'numInteractionPoints' in my dataset during training model - I binned this feature in order to avoid overffing , first top bar chart is from ...
2
votes
0answers
44 views

Multilabel Classification - Overfitting?

My task is the following: To input drug combinations and output renal failure-related symptoms from the drug combinations. Both the drug combinations and renal-failure related symptoms are represented ...
2
votes
0answers
71 views

Bias Formula in Machine Learning expanded using ground truth

Why is Bias calculated for $f(x)$? Shouldn't it be calculated for $Y$ (which is $f(x)$ + Noise $\epsilon$)? We are fitting our model to $Y$, So shouldn't we be calculating bias wrt to $Y$? Also, I ...
2
votes
0answers
319 views

Overfitting in K-means

How do you test your results for overfitting in a k-means run? Some people have said use a training set. I have about 1500 records and about 20 fields.
2
votes
0answers
175 views

How to improve a model with a high cross validation score yet with low accuracy on unseen data?

I have a model that is based on an experiment collected on 100 subjects. We are testing the model as follows: Record raw data from the subjects For each subject, compute the feature from the raw data ...
2
votes
0answers
75 views

Is my model overfitting when I add new features?

I'm working on simple 2-class classification problem. Nearly all features we have used (except one) are about the same for both classes: A random forest classifier confirms that one feature has an "...
2
votes
2answers
2k views

Minimum number of samples to train XGBoost without overfitting

When using Neural Networks for image processing I learned a rule of thumb: to avoid overfitting, supply at least 10 training examples for every neuron. Is there a similar rule of thumb for classifiers ...
2
votes
0answers
74 views

Overfitting and COLT/Statistical Learning Theory

The aspect of over-fitting is typically viewed from the perspective of both- accuracy and model complexity. To mitigate over-fitting, we usually have the practical approach of having k-fold ...
2
votes
0answers
53 views

information leakage when using empirical Bayesian to generate a predictor

Consider the following problem: I want to predict the next bat of a set of baseball player. I have a training data set, where it contains the historical bat records (0-1 encoded, which is our target ...
1
vote
0answers
10 views

model tuning by using loss curves

I have been practicing with the following dataset: http://archive.ics.uci.edu/ml/datasets/Concrete+Compressive+Strength for building a prediction model based on a MLP, but I have some doubts if the ...
1
vote
0answers
33 views

What is the impact of adding a layer in neural networks?

I was playing with hyper-parameters on https://playground.tensorflow.org/ using spiral dataset (classification). So , first I trained a network with 2 hidden layers and the final test and train loss ...
1
vote
0answers
26 views

Autoencoder fails to reconstruct

I'm trying to use an autoencoder to reduce dimensionality of my features. My features are of dimension 2048. I tried to train an autoencoder to reduce the dimensionality to 50. I'm using a single ...
1
vote
0answers
18 views

Escaping from overfitting hell: introducing regularization vs increasing training data

I am trying to identify noisy intervals in geomagnetic data using logistic regression, working with scikit-learn. Here is a typical spectrum of the data that I am working with: In this example, the ...
1
vote
0answers
74 views

Data augmentation for recommendation systems

I have a user-item matrix that I use to train a denoising autoencoder to predict the top-k items to recommend to the different users. The idea is to corrupt the matrix by erasing a percentage ...
1
vote
0answers
26 views

Do non-parametric models always overfit without regularization?

Let's scope this to just classification. It's clear that if you fully grow out a decision tree with no regularization (e.g. max depth, pruning), it will overfit the training data and get full accuracy ...
1
vote
0answers
43 views

Removing duplicate records before training

I am currently working on a project classifying text into classes. The specific problem is classifying job titles into various industry codes. For example "McDonalds Employee" might get classified to ...
1
vote
0answers
12 views

Oversampling for regression for data grouped in clusters

I am dealing with a regression problem in which I want to predict the upcoming value of a time-dependent variable by using the previous values of other variables (not including the output variable ...
1
vote
0answers
24 views

How to deal with training set that overfits very easily

I have a dataset consisting of 72 one-hot encoded (thus binary) features and 2.5K training examples. With this I am trying to solve a 10-class classification problem. My main problem is that no ...
1
vote
2answers
40 views

Will oversampling help with generalization (small imbalanced dataset)?

I have an imbalanced dataset (2:1 ratio) with about 60 patients and 80 features. I performed Recursive Feature Elimination (RFE) and stratified cross validation to reduce the features to 15 and I get ...
1
vote
0answers
44 views

How to interpret training results

I am working on an image similarity network. I have around 90,000 pairs of images contain an equal number of positive and negative samples. For learning the similarity between image pairs, I used the ...
1
vote
0answers
31 views

Validation score during training and checkpoint is different in keras

I have a tabular data with about 1500 columns where every column except the 1st column is sparse. I am trying to train a Feedforward neural network (1 hidden layer with 32 neurons) for a binary ...
1
vote
0answers
55 views

Conv Net Model is overfitting

So I made a convolution neural network to classify between different phonemes. My input datasets are a series of 0.4-second long spectrograms, the labels are each an individual phoneme that happens at ...
1
vote
0answers
168 views

High Variance on CNN

I'm using a shallow CNN for my current project [this one]. I have a training dataset consisting of 1000 samples and a test dataset of 400 samples. I'm using the test dataset to choose the best ...
1
vote
0answers
233 views

Remedies to CNN-LSTM overfitting on relatively small image dataset

Notes Using a pretrained model, trying data augmentation (not possible knowing nature of images, lowering number of parameters in the network, all didn't help) Context I have a sequence of images. ...
1
vote
0answers
60 views

Network either overfits or underfits, but never generalizes - what to do?

I have a simple network with 1st level an LSTM, dropout, fully-connected and softmax layers; loss is cross-entropy (four classes, well balanced). Sequence length to LSTM is 172 samples, data is z-...
1
vote
0answers
446 views

Keras - Why Validation data produce good results, while unseen data is performing poorly

I've built a feedforward net that predicts 2 classes (0,1). I've used the validation_split attribute like so: ...
1
vote
0answers
64 views

Why is my predicted vs observed plot worse for training than validation. Running an overfitted GBM on a binomial outcome

I have a binomial outcome that I am trying to predict using a gbm in h2o. I have set quite a low min_rows value for each node and it appears to be overfitting. See plots below. When I group the ...
0
votes
1answer
31 views

How many features do I select when doing feature selection for regression algorithms? Is R2 and RMSE good measures of success for overfitting?

Context: I'm currently crafting and comparing machine learning models to predict housing data. I have around 32000 data points, 42 features, and I'm predicting housing price. I'm comparing Random ...
0
votes
0answers
27 views

My Stacked LSTM seems to be doing worse than a shallower one

I started with a two layer LSTM (+ Dense Layers) and which was: ...
0
votes
0answers
24 views

Overfitting in imbalanced dataset

I am working on a dataset related to an insurance company and the objective is to predict if the insurance buyer will claim their travel insurance or not. Training data: https://raw.githubusercontent....
0
votes
1answer
52 views

Number of units for first layer in Keras Sequential Model

I have a huge CSV structured dataset. I'm feeding that dataset to a Keras Sequential Model. My question is, can my Model have number of units greater than the number of input features? At the moment, ...
0
votes
0answers
13 views

Strange behavior of CNN when forecasting time series

I have a time series containing 5 features. I tried to use LSTM to predict the next 112 periods in the series. However, I got very bad results. So I tried to use CNN. First, it did not work properly ...
0
votes
0answers
71 views

Reduce overfitting in a CNN model

We are Data science students and we are building a CNN model to pneumonia classification (dataset: https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia ). We have applied a data augmentation ...
0
votes
0answers
27 views

SVM overfitting with consistent validation results

I have some imbalanced (1400 samples of which 250 are +ve) data for a binary classification problem and I am running an SVM grid search optimising for precision. I am trying 3,4,5,6,7,and 8 stratified ...
0
votes
1answer
34 views

Training Object Detection model on just 10 images

I am trying to train an object detection model using Mask-RCNN with Resnet50 as backbone. I am using the pre-trained models from PyTorch's Torchvision library. I have only 10 images that I can use to ...
0
votes
1answer
32 views

Regularization hyperparam tuning during training

I have an idea for a regularization-hyperparam selection method, which I haven't encountered before and can't find on Google, but I'm sure someone has already tried it and I'm wondering what are the ...
0
votes
0answers
102 views

SVM is taking too long for hyperparameter tuning

I am running SVM,Logistic Rregression and Random Forest on the credit card dataset. My training dataset has the shape (454491, 30). I performed 5-fold cross validation(which took more than an hour) ...
0
votes
0answers
24 views

IIoT Sound Classification with Little Data

[Problem Statement] I have been working on a sound a classification problem with less 200 sound files (wav format) with the following imbalanced class distribution: ...
0
votes
1answer
41 views

How can I find if it is an overfitting problem?

I am new in Machine learning, and I want to detect emotions from the face. Preprocessing: I used equalizeHist to equalizes the histogram of grayscale images (JAFFE database with 213 images), in the ...
0
votes
0answers
38 views

What is cross validation good for, exactly?

I keep seeing that cross validation is a good way to reduce overfitting, but, in my case, I don't see how it helps much. Let me explain: I'm interested in running a Multinomial Logit for prediction, ...
0
votes
0answers
25 views

CNN is not learning anything

I'm training a CNN network to detect relations between entities in written texts. I am suffering from an overfitting problem, I have high accuracy and low loss at the training step, but my model can't ...
0
votes
0answers
40 views

ConvNet - What to improve regarding architecture, procedure and technique?

I have a dataset of 180k images of license plates (so, not necessary to localize the license plate at first) for which I try to recognize the characters on the images (License plate recognition). All ...
0
votes
0answers
20 views

Why Overfitting sometimes appears when compile model multiple time, is it normal?

At the time I got small datasets of brainwaves (EEG) (105 samples) for 3-class classification problem. I split my data into 3 part: Train data = 90 (data) Validation data = 10 (data) Test data = 5 (...
0
votes
0answers
76 views

weight decay in ResNet50

Can someone please guide for implementing weight decay in transfer learning approach? I want to regularize the pre-trained model ResNet50, where I'm fine-tuning the model for an image classification ...
0
votes
0answers
10 views

Why is my time series model predicting strange results?

I am trying to predict some time-series data. The output data predicts two numbers (one that's usually greater than 1 and another that is usually less than 1). I've plotted about 800 samples where the ...