Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

Filter by
Sorted by
Tagged with
0
votes
1answer
33 views

How many features do I select when doing feature selection for regression algorithms? Is R2 and RMSE good measures of success for overfitting?

Context: I'm currently crafting and comparing machine learning models to predict housing data. I have around 32000 data points, 42 features, and I'm predicting housing price. I'm comparing Random ...
1
vote
1answer
32 views

Derive features from test-set?

I have a dataset of choices (between A,B and C) done by certain users, and I want to train a neural network to predict the choices. I divide in train and test sets. An instance is formed by: [UserId, ...
0
votes
0answers
28 views

My Stacked LSTM seems to be doing worse than a shallower one

I started with a two layer LSTM (+ Dense Layers) and which was: ...
1
vote
1answer
18 views

NGBoost and overfit - which model is used?

While training an NGBoost model I got: ...
7
votes
2answers
242 views

Dropping features after final evaluation on test data

Would you please let me know if I am committing a statistical or machine learning mal-practice in this procedure? I want to estimate meteorological variable y1 from ${x_1, ..., x_{10}}$ variables. I ...
1
vote
1answer
44 views

Hyperparameter tunning for Random Forest- choose the best max depth

I'm trying to choose the best parameters for random forest model. For that goal I hae run my model in loop with only one parameter and each time I have changed the number for the parameter max depth. ...
4
votes
2answers
144 views

Trying to figure out which the training set is

Can someone help me on this one? As can be seen in the screenshot, it says the loss is 1/2. Where's that 1/2 coming from? How can I replace the values in the h(s) function? Source PDF
0
votes
0answers
24 views

Overfitting in imbalanced dataset

I am working on a dataset related to an insurance company and the objective is to predict if the insurance buyer will claim their travel insurance or not. Training data: https://raw.githubusercontent....
0
votes
1answer
50 views

Python: How to test a RandomForest regression model for Overfitting?

I'm a beginner in this area so maybe I'm doing something wrong here. I'm using RandomForest for a regression model and wanted to see if my model is overfitting. Here is what I did: EDIT: I use ...
0
votes
1answer
53 views

Number of units for first layer in Keras Sequential Model

I have a huge CSV structured dataset. I'm feeding that dataset to a Keras Sequential Model. My question is, can my Model have number of units greater than the number of input features? At the moment, ...
0
votes
1answer
43 views

why should i do target encoding within cv loop?

i wish to use target encoding, using the category encoders sklearn library. I don't really understand why it is necessary to include this as a step in a sklearn pipeline WITHIN the cross validation ...
0
votes
0answers
13 views

Strange behavior of CNN when forecasting time series

I have a time series containing 5 features. I tried to use LSTM to predict the next 112 periods in the series. However, I got very bad results. So I tried to use CNN. First, it did not work properly ...
0
votes
0answers
71 views

Reduce overfitting in a CNN model

We are Data science students and we are building a CNN model to pneumonia classification (dataset: https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia ). We have applied a data augmentation ...
3
votes
2answers
569 views

Over-sampling: is my model over-fitting?

I would like to ask you some questions on how to consider (good or not) the following results: ...
2
votes
1answer
30 views

machine learning disasters

I am writing a research paper and I am looking for reliable sources that provide information on disasters of machine learning. Especially in the field of autonomous driving. Have there been any ...
7
votes
1answer
269 views

Who invented the concept of over-fitting?

I list the references that I found so far. Shortly, the first appearance of the term was in 1670, first appearance in in close meaning was in 1827, first appearance in a biological paper was in 1923 ...
0
votes
1answer
34 views

Why use regularization?

In a linear model, regularization decreases the slope. Do we just assume that fitting a lin model on training data overfits by almost always creating a slope which is higher than it would be with ...
0
votes
1answer
32 views

Why might my validation loss flatten out while my training loss continues to decrease?

In my effort to learn a bit more about data science I scraped some labeled data from the web and am trying to classify examples into one of three classes. I am running into a problem that, regardless ...
0
votes
0answers
27 views

SVM overfitting with consistent validation results

I have some imbalanced (1400 samples of which 250 are +ve) data for a binary classification problem and I am running an SVM grid search optimising for precision. I am trying 3,4,5,6,7,and 8 stratified ...
0
votes
1answer
37 views

Compare cross validation and test set results

I am having a hard time understanding the results of a cross validation test and a test run on a test set. First I made the following pipeline: ...
2
votes
1answer
60 views

Normal distribution and Random Forest

I have big table in dataframe (600k rows) which has y column (the variable I want to predict) and other 4 other columns that are the X. I have run RF regressor and I got score of 0.87 when I run it ...
4
votes
5answers
2k views

Why does my model produce too good to be true output?

I am trying to run a binary classification problem on people with diabetes and non-diabetes. For labeling my datasets, I followed a simple rule. If a person has T2DM...
0
votes
1answer
34 views

Training Object Detection model on just 10 images

I am trying to train an object detection model using Mask-RCNN with Resnet50 as backbone. I am using the pre-trained models from PyTorch's Torchvision library. I have only 10 images that I can use to ...
2
votes
2answers
36 views

Why is large decision tree likely to overfit

My lecture slide told me that if we don't prune the regression tree, then the tree likely to over-fit. So, I wonder why would that happen? Is that because if the tree grows too large, we would end up ...
2
votes
1answer
35 views

Why Continous Variable Buckets Overfitting model

I have a continuous (high cardinal discrete) variable 'numInteractionPoints' in my dataset during training model - I binned this feature in order to avoid overffing , first top bar chart is from ...
3
votes
1answer
223 views

Determining whether a Machine Learning model is overfitted with regard to the stability of the features

I need to know how would I get to know if I have overfitted my Machine Learning model on the train data. The performance metric I have used is Logistic Loss. Does the stability of the features affect ...
1
vote
0answers
10 views

model tuning by using loss curves

I have been practicing with the following dataset: http://archive.ics.uci.edu/ml/datasets/Concrete+Compressive+Strength for building a prediction model based on a MLP, but I have some doubts if the ...
5
votes
2answers
197 views

Why an increasing validation loss and validation accuracy signifies overfitting?

When I train a neural network, I observe an increasing validation loss, while at the same time, the validation accuracy is also increased. I have read explanations related to the phenomenon, and it ...
1
vote
0answers
33 views

What is the impact of adding a layer in neural networks?

I was playing with hyper-parameters on https://playground.tensorflow.org/ using spiral dataset (classification). So , first I trained a network with 2 hidden layers and the final test and train loss ...
4
votes
1answer
389 views

Can a novelty detection model overfit?

Can a novelty detection model overfit? In novelty detection, the model is trained on normal data instances (not polluted by outliers) where no labels are used in the training process, while validated ...
6
votes
0answers
68 views

XGBoost skews towards minority class

I have a dataset with 85k positive labels and 53k negative labels. For this use-case, I am trying to maximize my efforts to the negative class (accurately identify true negatives, and minimize false ...
0
votes
1answer
31 views

Why does the overfitting decreases if we choose K to be large in K-nearest neighbors?

I am studying machine learning and I am focusing on K-nearest neighbors . I have understood the algorithm, but I have still a doubt, which is on how to choose the K for the number of neighbors. I ...
-1
votes
1answer
28 views

How we can identify the problem of Overfitting and underfitting and maintain bias?

Basically, I'm new to the data science field, and I'm getting a little bit of confusion about overfitting and underfitting. Are overfitting and underfitting is ...
0
votes
1answer
32 views

Regularization hyperparam tuning during training

I have an idea for a regularization-hyperparam selection method, which I haven't encountered before and can't find on Google, but I'm sure someone has already tried it and I'm wondering what are the ...
0
votes
0answers
111 views

SVM is taking too long for hyperparameter tuning

I am running SVM,Logistic Rregression and Random Forest on the credit card dataset. My training dataset has the shape (454491, 30). I performed 5-fold cross validation(which took more than an hour) ...
1
vote
1answer
33 views

How do you identify whether your RMSE score is good or not?

Im building a XGBoost regression model to predict the values in the range of -3 to 3. Im using Root Mean Squared Error to evaluate the model. With hyper-parameter tuning and everything the best scores ...
2
votes
0answers
44 views

Multilabel Classification - Overfitting?

My task is the following: To input drug combinations and output renal failure-related symptoms from the drug combinations. Both the drug combinations and renal-failure related symptoms are represented ...
0
votes
2answers
44 views

100% accuracy on both train and test after feature engineering

The original dataset is of ~17K compound structures almost equally divided with labels indicating yes or no, after heavy use of mol2vec and rdkit I have created ~300 datapoints Using the boosted trees ...
-1
votes
1answer
36 views

How to identify Overfitting in RandomForestClassifier?

Im building a sentiment classification model using RandomForestClassifier. I got the training accuracy of 99.65 & cross-validation( RepeatedStratifiedKFold-5 folds) accuracy of 97.29. I used f1 ...
0
votes
0answers
24 views

IIoT Sound Classification with Little Data

[Problem Statement] I have been working on a sound a classification problem with less 200 sound files (wav format) with the following imbalanced class distribution: ...
0
votes
2answers
79 views

Is my model overfitting?

I'm currently building my first model with sklearn to predict whether a customer will renew a subscription. I'm using a random forest because I've heard that they are robust to overfitting. The ...
0
votes
1answer
41 views

How can I find if it is an overfitting problem?

I am new in Machine learning, and I want to detect emotions from the face. Preprocessing: I used equalizeHist to equalizes the histogram of grayscale images (JAFFE database with 213 images), in the ...
0
votes
1answer
33 views

model selection in clustering

I am working on a mall customer segmentation dataset (5 features, 200 rows) using clustering. This dataset does not have any ground truth labels. I had a few doubts regarding clustering: Can I use ...
12
votes
3answers
1k views

Overfitting in Linear Regression

I'm just getting started with machine learning and I have trouble understanding how overfitting can happen in a linear regression model. Considering we use only 2 feature variables to train a model, ...
1
vote
2answers
114 views

Does gradient boosting algorithm error always decrease faster and lower on training data?

I am building another XGBoost model and I'm really trying not to overfit the data. I split my data into train and test set and fit the model with early stopping based on the test-set error which ...
1
vote
0answers
27 views

Autoencoder fails to reconstruct

I'm trying to use an autoencoder to reduce dimensionality of my features. My features are of dimension 2048. I tried to train an autoencoder to reduce the dimensionality to 50. I'm using a single ...
0
votes
0answers
38 views

What is cross validation good for, exactly?

I keep seeing that cross validation is a good way to reduce overfitting, but, in my case, I don't see how it helps much. Let me explain: I'm interested in running a Multinomial Logit for prediction, ...
0
votes
0answers
26 views

CNN is not learning anything

I'm training a CNN network to detect relations between entities in written texts. I am suffering from an overfitting problem, I have high accuracy and low loss at the training step, but my model can't ...
1
vote
0answers
18 views

Escaping from overfitting hell: introducing regularization vs increasing training data

I am trying to identify noisy intervals in geomagnetic data using logistic regression, working with scikit-learn. Here is a typical spectrum of the data that I am working with: In this example, the ...
6
votes
2answers
441 views

Can a linear regression model without polynomial features overfit?

I've read in some articles on the internet that linear regression can overfit. However is that possible when we are not using polynomial features? We are just plotting a line trough the data points ...

1
2 3 4 5