Questions tagged [training]

Training is the part of machine learning whereby a model is "trained" on a define portion of a dataset to learn attributes and statistical features of the data. It's counterparts are called Testing and Validation. After training a model is tested and validated on another portion of the dataset.

Filter by
Sorted by
Tagged with
2
votes
2answers
84 views

Dataset and why use evaluate()?

I am starting in Machine Learning, and I have doubts about some concepts. I've read we need to split our dataset into training, validation and test sets. I'll ask four questions related to them. 1 - ...
0
votes
1answer
33 views

K-Fold cross validating with random forest - how to correctly fit model to every fold?

So I have created K-Folds from my data using this code: ...
0
votes
0answers
18 views

Is it possible to perform additional learning on a pre-trained model with YOLO based on Darknet

I would like to have a capability of my object detection model to improve over time as the available dataset grows over time, but avoid re-training a model from scratch every time I want to update the ...
0
votes
1answer
102 views

Train a spaCy model for semantic similarity

I'm attempting to train a spaCy model for the purposes of computing semantic similarity but I'm not getting the results I would anticipate. I have created two text files that contain many sentences ...
0
votes
0answers
29 views

Validation loss curve - store all losses as a list [closed]

I'm very new in this field. The original code is from this GitHub: https://github.com/YuliaRubanova/latent_ode (run_models.py) file. I want to draw a learning curve using the validation loss. The code ...
0
votes
0answers
24 views

Test data relevance to a model (covariate shift)

I am trying to design an algorithm that will allow to calculate the relevance of test data to a trained model. This can be done by checking if predictor variables have a different distribution in ...
0
votes
0answers
10 views

Loss semi-regularly spikes only with no regularization

I'm fairly new to neural networks, and I'm seeing some behavior during training I don't understand. I'm working with a pretty straightforward feedforward neural net classifier with 5 hidden layers, ...
1
vote
0answers
9 views

How to make an DL model predict Correctly [closed]

So I trained a DL algorithm using Keras for Human Action Recognition. The model has an accuracy of about 85 percent and a loss of 0.3 something. The problem is that the model did not predict well on ...
0
votes
1answer
25 views

Error with MSE in LSTM

I'm trying to fit an LSTM model on my dataset, using also a validation set. My datasets have the following shapes: ...
0
votes
0answers
11 views

How can I build my voice speech-to-text model?

I found an instruction to build such kind of custom model on Azure. Prepare data for Custom Speech However, I would like to either fine-tune or transfer learning on Google Colaboratory or docker. In ...
1
vote
1answer
39 views

test data is not a good representation of train data

I have predefined train and test sets. On generating some statistics like value_counts and checking the unique values, I feel that there is a 'lot' of difference between the distributions of the ...
1
vote
1answer
38 views

Poor binary text classification results

I have a binary NLP classification task to identify text that talks about a target topic from millions of sentences. Between 5-10% of sentences are positive, the rest is negative. I have trained ...
4
votes
1answer
132 views

What changes is the Neural Network back-propagation algorithm doing on the weights?

I have seen the formula for back-propagation algorithm for neural network error minimization, but I am not quite sure about what changes it is performing on the weights individually. Let us suppose a ...
1
vote
0answers
72 views

How to train and evaluate model based on inner ordering of data subsets?

I've got a problem which I don't know how to frame properly (what techniques to use, what data structure, etc). Here's a rough definition of it: ...
0
votes
0answers
15 views

Model accuracy and validation accuracy stuck at constant value for denoising autoencoder model

I am building a denoising autoencoder (DAE) to denoise respiratory signals. I pass through the model both noisy and clean versions of the signal (in frame sizes as multiples of 1024). I've set up my ...
0
votes
1answer
21 views

ValueError: Layer model_4 expects 1 input(s), but it received 10 input tensors

I've directory structure like this, for my dataset: |--train |--test |--valid In the train folder, these are pairs of images like xyz_sat and xyz_mask. So I've ...
0
votes
0answers
12 views

validation loss early increase (during warm-up)

Several questions have been asked about validation loss behavior during training of a DNN. It's clear to me that validation loss and accuracy are somehow correlated, but their curves can differ from ...
0
votes
1answer
28 views

How to use train_test_split with existing dataset?

I am looking for an example of how to use train_test_split with an existing dataset. I have a CSV that can be bought into a dataset with: ...
0
votes
0answers
9 views

Neural network training time

I am very new to NN and I have a dataset that I use NN to map the relation between its inputs and output using MATLAB and I will test different NN architectures and algorithms. Regarding training time,...
0
votes
0answers
6 views

Which tool do you use for creating continuous training pipelines for MLOPS?

One key component of MLOPS is continuous training. Which means the end to end training is put in a pipeline which can be triggered, versioned and metadata of the pipeline can be tracked. Thus enabling ...
0
votes
0answers
20 views

Different AUC values for xgb and sklearn built in functions [duplicate]

The model is trained with early stopping on a validation set: ...
1
vote
0answers
16 views

Q: Training a CNN-LSTM on video inputs

Hello everyone! I implemented the following model, for action classification from videos, where each frame is 224x224x3, a video consists of ...
0
votes
2answers
60 views

How can I get the type of fitting in this curve?

Does that overfitting ? How can I interpret the curve ?
1
vote
0answers
19 views

Should I reshuffle the training set when benchmarking neural networks?

I'm trying to set up a fair benchmarking between various RNN models, where each of them is trained until convergence with a fixed random seed. Because the task is very costly, I am only able to run ...
0
votes
0answers
10 views

How to distribute train and test set equally in the matter of low and fast growth cases?

I'm working on a project which we are using python (Random Forest Regression) predicting tumor doubling time. when I plot different test sets I found that my data is not distributed equally into train ...
1
vote
0answers
18 views

Different data dimension when indexing with R

I am facing on something that I cannot explain, so may be someone could explain me. I have a dataset which contain 102011 data. I want to take 70% for the train and 30% remaining for the validation. I ...
0
votes
0answers
8 views

Training the input to a neural network

I have a network that takes as input 3 numbers and outputs 2 and I trained it on a given data set, to predict the 2 numbers. Now I would like to freeze all the layers of the network, and make the ...
0
votes
1answer
35 views

Best Way to find the important features for the model [duplicate]

I have data with 245 Features and almost all of the features are categorical. I would like to know what will be the best approach to find the important features for training the model. I know I can ...
0
votes
1answer
12 views

Assess the goodness of a ML generative model (text)

Take a RNN network fed with Shakespeare and generating Shakespeare-like text. Once a model seems mathematically fine, as can be assessed by observing its loss and accuracy over training epochs, how ...
0
votes
1answer
17 views

Train Test Split procedure

Given that the sample size is small (roughly 2,700 observations), I wanna do a multiclass classification. Should I use the full sample instead of the train test split?
0
votes
0answers
11 views

Using mathematical derivatives of input data to augment training input data

I'm thinking of how to design a basic feedforward neural network that would be able to predict future datapoints given past datapoints. I'm very new to neural network design so I'm wondering if there'...
0
votes
0answers
18 views

Why does my training explode?

I'm trying to implement A new lower and upper bound estimation model using gradient descend training method for wind speed interval prediction For simplicity purposes, I've changed the training data. ...
2
votes
0answers
48 views

Multi-Core CPU training on Keras

I want to train models on a machine with multi-cores, I know training on GPU is better but I only have access now on CPU. Which parameters should I set using keras.models.Model.fit to utilize all ...
1
vote
0answers
12 views

Are less training epochs better in the following scenario

So I have a scenario in which the training data is being generated in response to what the Neural Network backed actor is doing. In essence its giving feedback to the Neural Network based on all of ...
0
votes
0answers
9 views

How to deal with errors or inconsistencies in the training data?

There are inconsistant wrong labels and consistant errors in training data. For the former I tried MC-dropout and data Shapley. For the later I wonder if manual data curation is a requisite?
0
votes
0answers
18 views

Generating unique points with an auto-encoder

I have been working on some research using a type of auto-encoder to generate new points with specific desirable properties. I trained my network and successfully generated some points, but when I ...
0
votes
0answers
19 views

pytorch lightning produces no checkpoint when learning rate fine tuning ison

My problem is concerning with using the automatic learning rate finder of pytorch lightning. In case I use this feature there isn't any checkpoint output produced at any time during the training of ...
0
votes
1answer
629 views

1D target tensor expected, multi-target not supported

I am trying to train my model. My model outputs a [4,2] tensor where 4 is the batch size and 2 because of binary classification. After receiving the outputs I found the index of the maximum element ...
0
votes
0answers
41 views

Custom stratification for train and test

The data contains half-year amounts of sales. The train dataset would be fit on the second half of 2020 and the test dataset would be fit on the first half of 2021. There are 2 columns containing this,...
0
votes
0answers
11 views

Validation output in a custom training loop not working - Tensorflow

I am new to Deep Learning and I am trying to learn more about implementation in Tensorflow and Keras. I am basing my work on this link : https://www.tensorflow.org/guide/keras/...
0
votes
1answer
16 views

Aren't balanced data sets important in regression?

Why is it that the necessity for balanced data sets is (almost) always exclusively mentioned in the context of classification but not of regression?
2
votes
2answers
51 views

Spliting Training Test and Validation for Image Dataset

I have 600 images in the training folder, 200 images in the validation folder, and 200 images in the test folder. Suppose if I fit the training data generator and validation data generator for some ...
0
votes
2answers
25 views

No target response variable in my testing data

I have two datasets which are the training and testing set. The training data has a target variable, but the testing set does not. What should I do to fix the issue with the testing set?
0
votes
0answers
32 views

How can I check a confusion_matrix after fine-tuning with custom datasets?

Background I would like to check a confusion_matrix, including precision, recall, and f1-score like below after fine-tuning with custom datasets. Fine tuning process and the task are Sequence ...
1
vote
1answer
96 views

How train - test split works for Graph Neural Networks

I have recently started studying GNN's. I have covered GCN and GraphSage so far. But I am confused regarding the process when testing occurs. Now suppose in the graph above I am using the nodes as ...
0
votes
1answer
31 views

Does validation data has any effect on training or it acts solely without affecting the training?

When using Keras library of Python, we use validation data with training data while training our model. In every epoch, we get a validation accuracy. Does this validation accuracy have any effect on ...
0
votes
0answers
13 views

Weight for Samples on SVM

there is a option sample_weight in fit(X[, y, sample_weight]) function (OneClassSVM, sklearn library). If I use the option ...
1
vote
0answers
45 views

How to do online retraining of model on a single new data point/observation?

I am trying to investigate the effect on performance on old data and new data when a classifier is retrained on only the new observation when it is encountered. The aim is to retrain the classifier on ...
2
votes
0answers
24 views

When to stop the final model training?

Let's say I'm participating in a Kaggle image recognition competition. Firstly, I create a train/validation split and find the good hyperparameters for my model. Here the stopping criterion is when ...
0
votes
1answer
24 views

how to correct mislabeled data in training, validation and test set

In an image classification task, I know there are mislabeled data. should I remove/correct them in all training / validation / test set ? I saw this article https://arxiv.org/pdf/2103.14749.pdf but I ...

1
2 3 4 5
12