Questions tagged [validation]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
2answers
16 views

dataset split for image classification

I am trying to do image classification for 14 categories (around 1000 images for each cat). And i initially created two folders for training and validation. In this case, do I still need to set a ...
0
votes
0answers
25 views

Validation accuracy does not increase for binary classification using GNN

I am trying to perform graph classification with GINConv model of GNN. I have tried everything from varying dropouts to weight decay (for L2 regularization), learning rate (1e-6 to 1e-3), batch ...
0
votes
0answers
12 views

Validation Loss Decreases Then Increases to The Same Values

I am training a transformer-based model using Pytorch. The training loss decreases until it hits a floor, which is expected. However, the validation loss decreases to a minimum then starts increasing. ...
0
votes
0answers
25 views

Three different errors using external information. Which one makes sense? (Or how to interpret each?)

My goal is to compare clustering methods considering different method and different number of clusters using an external information. Could anyone please give some opinion/ recommend book/paper about ...
0
votes
0answers
50 views

Time series imputation benchmark

In a work, I have to benchmark different algorithms to fill in missing values in time series. I insist on the fact that this is imputation and not forecasting. In my case, I have access to 15 years of ...
2
votes
0answers
14 views

Validation set after hyperparameter tuning [duplicate]

Let's say I'm comparing few models, and for my dataset I'm using train/validation/test split, and not cross validation. Let's say I'm completely done with parameter tuning for one of them and want to ...
0
votes
1answer
20 views

How to compare a machine learning model and a rule based model

The difference between machine leaning models and rule based model is that you feed input and output to machine learning models to get rules where you feed input and rules to rule based models to get ...
0
votes
1answer
62 views

How is the validation set processed in PyTorch?

Say, one uses the MNIST dataset and splits the provided training data of size 60,000 into a training set (50,000) and a validation set (10,000). The provided test data of size 10,000 is used as the ...
1
vote
1answer
42 views

Does overfitting depend only on validation loss or both training and validation loss?

There are several scenarios that can occur while training and validating: Both training loss and validation loss are decreasing, with the training loss lower than the validation loss. Both training ...
2
votes
1answer
35 views

Using the whole dataset for testing (not validation) in case of small datasets

for an object detection task I created a small dataset to train an object detector. The class frequency is more or less balanced, however I defined some additional attributes with environmental ...
3
votes
1answer
79 views

Why do machine learning engineers insist on training with more data than validation set?

Among my colleagues I have noticed a curious insistence on training with, say, 70% or 80% of data and validating on the remainder. The reason it is curious to me is the lack of any theoretical ...
1
vote
1answer
16 views

Is it right to maintain the train distribution in test set for unbalanced data?

If the training set was unbalanced the chances are the model will be biased. But if the data distribution in the test set is the same distribution as the train set, this kind of bias is not going to ...
0
votes
0answers
48 views

How can realize the evaluation/validation of unsupervised models through unlabeled data?

I'm researching anomaly detection, which is nothing else than outliers detection on a set of time-series web servers access log data or network traffic. Recently I re-faced to following fundamental ...
0
votes
2answers
46 views

Validation loss and validation accuracy stay the same in NN model

I am trying to train a keras NN regression model for music emotion prediction from audio features. (I am a beginner in NN and I am doing this as study project.) I have 193 features for training/...
0
votes
0answers
43 views

How to set thresholds to automate the management of model drift?

I have a trained ML Model and new data coming in every week. The new data sometimes vary too much (in a statistical sense) and, therefore, the performance of the ML model degrades. The data-set schema ...
0
votes
1answer
37 views

Augmentation on test dataset and validation dataset

I'm training a segmentation model (computer-vision). Thus, my dataset contains images and masks (binary segmentation of objects). I'm augmenting the training dataset (applying random crop, rotation or ...
0
votes
0answers
12 views

Appropriate naive benchmark for class recall in binary classification for unbalanced dataset

I have an unbalanced dataset with 3969 rows of customer data. The labels are whether or not they subscribed for a loan (yes or no). There are 3618 no cases (91.2%) and 351 yes cases (8.8%). I am more ...
0
votes
1answer
96 views

SMOTE train test split with validation data [duplicate]

Would like to ask, in which way to use SMOTE? My dataset is imbalanced and a multiclass problem. As I read in many posts, use SMOTE method only for the training dataset (X_train and y_train). Not for ...