Questions tagged [training]
Training is the part of machine learning whereby a model is "trained" on a define portion of a dataset to learn attributes and statistical features of the data. It's counterparts are called Testing and Validation. After training a model is tested and validated on another portion of the dataset.
682
questions
1
vote
0
answers
18
views
Should the model be defined again before training it to new data?
I wanted to fit the LSTM model on new data set in a loop so I have implemented it like this
...
- 157
1
vote
0
answers
51
views
How to include the sudden peaks/bursts in LSTM based time-series model's training
I am using LSTM for time-series prediction whereby I am taking past 50 values as my input. Now, the thing is that it is predicting just OKish, and not doing the exact prediction, especially for the ...
- 131
0
votes
3
answers
169
views
How to do modelling for pairs of non i.i.d. data?
I have a dataset in which i have the labels for candidates on whether they would be hired,interviewed_and_failed,not_interviewed_at_all. The task is to predict for new jobs/new candidates what these ...
- 73
3
votes
3
answers
127
views
What can be done about mislabeled data points in the training set of a binary classification model?
The training set is exposed to the labels, but what if a portion of these labels were incorrectly labeled by the curator? Is there a way of searching for these training set examples?
- 185
1
vote
3
answers
300
views
0
votes
1
answer
26
views
How to address label imbalance in deciding train/test splits?
I'm working on a dataset that isn't split into test and train set by default and I'm a bit concerned about the imbalance between the 'label' distributions between them and how they might affect the ...
- 101
0
votes
1
answer
114
views
Is it good to use .fit to xtest when we use PolynomialFeatures() of sklearn?
My teacher did this in class, and I'm wondering is this ok to use .fit_transform with xtest? It shouldn't just be poly.transform(xtest)
Teacher's Code
...
- 103
0
votes
0
answers
17
views
Underlying explanation of Neural-Network basedtime-series future-value prediction
I have confusion related to Feed Forward Neural Network.
I train my network for time-series prediction, and it is working great and as expected.
I know how NNs train and to predict i.e. there is ...
- 131
1
vote
1
answer
207
views
PowerTransformer Producing Unexpected Result for Just One Column
I'm doing some preprocessing on my training data before fitting it to a model. Upon checking the results, there is one column that is returning 0 rather than 1 for the standard deviation. (all columns ...
- 11
0
votes
1
answer
26
views
Can we not backpropagate model
I saw a model based on CNN for question classification. The author said that they don't backpropagate gradient to embeddings. How this is possible to update network if you don't backpropagate please?
...
- 231
0
votes
1
answer
4k
views
cuDNN isn't found FWD algo for convolution. How to TRAIN DARKNET ON GE FORCE GTX 1650
ISSUE: while training Darknet with GE FORCE GTX 1650 using following:
CUDA 11.0
cuDNN 8.0.5
OPENCV 4.5
Model starts training with config file details as below for [net] section:
...
0
votes
1
answer
363
views
Is it correct to train and validate the model on F1-score metrics?
I am trying to do experiments on multiple data sets. Some are more imbalanced than others. Now, in order to assure fair reporting, we compute F1-Score on test data. In most machine learning models, we ...
- 138
1
vote
1
answer
285
views
Keras: How to restore initial weights when using EarlyStopping
Using Keras, I setup EarlyStoping like this:
EarlyStopping(monitor='val_loss', min_delta=0, patience=100, verbose=0, mode='min', restore_best_weights=True)
When I ...
- 113
1
vote
0
answers
27
views
How can one ensure the quality of collectively created ground truth labels?
Here is the situation: I have a small tool to assign (one or multiple) labels to image data. In the tool, you can not only assign a label(s) to the images but also assign a score for the certainty of ...
- 11
1
vote
1
answer
112
views
The behavior of the cross validation error and training error in underfitting case is not clear
I currently study the "Machine Learning" course on Coursera.org by Andrew Ng, it comes to a topic that discusses the performance of learning algorithms under different conditions.
Here, we ...
- 13
1
vote
1
answer
42
views
Random Forest Model Train, Save and Predict Later vs Train and Predict Right Away - Different Results
I tested two pieces of code and they delivered different results, which was quite unexpected.
First piece of code is supposed to train models in a k-fold manner, preserve each one of these fitted ...
- 39
1
vote
2
answers
116
views
Updating a train/val/test set
It is considered best practice to split your data into a train and test set at the start of a data science / machine learnign project (and then your train set further into a validation set for ...
- 448
1
vote
0
answers
206
views
Batch size and steps per epoch
My data size is 6011, which is a prime number, and therefore, the only batch size number that divides this data evenly is either 1 or 6011. However, I need the batch size to be 32, which means that ...
- 145
0
votes
2
answers
82
views
validation/test set uniqueness question
Hopefully a simple question, but it's a little unclear to me on how best to separate train/validate/test sets.
I have say 100 examples of class A. I'm classifying text into either class A, which I ...
- 115
-1
votes
3
answers
2k
views
Transfer learning on YOLOv5 for character and shape detection
The task is to detect rotated alphanumeric characters embedded on colored shapes. We will have an aerial view of the object (from a UAS: Unarmed Aerial System), something of this sort:
(One Uppercase ...
- 103
4
votes
3
answers
13k
views
What do x and y mean when working with data in the machine learning domain?
We need train & test data, so what do x and y mean?
Does it mean that it divides 15% to the 'x_train', 'x_val', 'y_train', 'y_val' ?
...
- 41
1
vote
0
answers
547
views
Features should be at same order in train and test?
I train ensemble model
Logistic regression
random forest
adaboost class
With pcr and random search
In pipeline.
The features in the train and test are Equal but not in the same order.
Will it be ...
- 35
0
votes
0
answers
22
views
For a binary classification algorithm, is there an objective way to know how large your set of positive and negative labels need to be?
We're training a binary classification algorithm using a combined total of 2000 positive and negative labels that we purchased from a data vendor.
We mostly used all the textbook machine learning ...
2
votes
2
answers
315
views
Dataset and why use evaluate()?
I am starting in Machine Learning, and I have doubts about some concepts. I've read we need to split our dataset into training, validation and test sets. I'll ask four questions related to them.
1 - ...
- 125
0
votes
1
answer
534
views
K-Fold cross validating with random forest - how to correctly fit model to every fold?
So I have created K-Folds from my data using this code:
...
1
vote
2
answers
629
views
Train a spaCy model for semantic similarity
I'm attempting to train a spaCy model for the purposes of computing semantic similarity but I'm not getting the results I would anticipate.
I have created two text files that contain many sentences ...
- 11
1
vote
0
answers
32
views
How to make an DL model predict Correctly [closed]
So I trained a DL algorithm using Keras for Human Action Recognition. The model has an accuracy of about 85 percent and a loss of 0.3 something. The problem is that the model did not predict well on ...
- 11
0
votes
1
answer
491
views
Error with MSE in LSTM
I'm trying to fit an LSTM model on my dataset, using also a validation set. My datasets have the following shapes:
...
- 53
1
vote
1
answer
263
views
test data is not a good representation of train data
I have predefined train and test sets. On generating some statistics like value_counts and checking the unique values, I feel that there is a 'lot' of difference between the distributions of the ...
- 13
1
vote
1
answer
102
views
Poor binary text classification results
I have a binary NLP classification task to identify text that talks about a target topic from millions of sentences. Between 5-10% of sentences are positive, the rest is negative.
I have trained ...
- 173
3
votes
1
answer
177
views
What changes is the Neural Network back-propagation algorithm doing on the weights?
I have seen the formula for back-propagation algorithm for neural network error minimization, but I am not quite sure about what changes it is performing on the weights individually.
Let us suppose a ...
user127052
0
votes
0
answers
76
views
How to train and evaluate model based on inner ordering of data subsets?
I've got a problem which I don't know how to frame properly (what techniques to use, what data structure, etc).
Here's a rough definition of it:
...
- 135
0
votes
1
answer
4k
views
ValueError: Layer model_4 expects 1 input(s), but it received 10 input tensors
I've directory structure like this, for my dataset:
|--train
|--test
|--valid
In the train folder, these are pairs of images like xyz_sat and xyz_mask.
So I've ...
- 129
0
votes
1
answer
424
views
How to use train_test_split with existing dataset?
I am looking for an example of how to use train_test_split with an existing dataset. I have a CSV that can be bought into a dataset with:
...
- 53
1
vote
0
answers
237
views
Q: Training a CNN-LSTM on video inputs
Hello everyone!
I implemented the following model, for action classification from videos, where each frame is 224x224x3, a video consists of ...
- 11
0
votes
2
answers
68
views
How can I get the type of fitting in this curve?
Does that overfitting ? How can I interpret the curve ?
- 1
1
vote
0
answers
34
views
Should I reshuffle the training set when benchmarking neural networks?
I'm trying to set up a fair benchmarking between various RNN models, where each of them is trained until convergence with a fixed random seed. Because the task is very costly, I am only able to run ...
- 111
1
vote
0
answers
19
views
Different data dimension when indexing with R
I am facing on something that I cannot explain, so may be someone could explain me. I have a dataset which contain 102011 data.
I want to take 70% for the train and 30% remaining for the validation. I ...
- 137
0
votes
1
answer
150
views
Best Way to find the important features for the model [duplicate]
I have data with 245 Features and almost all of the features are categorical. I would like to know what will be the best approach to find the important features for training the model. I know I can ...
- 193
0
votes
1
answer
28
views
Assess the goodness of a ML generative model (text)
Take a RNN network fed with Shakespeare and generating Shakespeare-like text.
Once a model seems mathematically fine, as can be assessed by observing its loss and accuracy over training epochs, how ...
- 143
0
votes
1
answer
44
views
Train Test Split procedure
Given that the sample size is small (roughly 2,700 observations), I wanna do a multiclass classification. Should I use the full sample instead of the train test split?
0
votes
0
answers
25
views
Using mathematical derivatives of input data to augment training input data
I'm thinking of how to design a basic feedforward neural network that would be able to predict future datapoints given past datapoints. I'm very new to neural network design so I'm wondering if there'...
- 9
2
votes
0
answers
561
views
Multi-Core CPU training on Keras
I want to train models on a machine with multi-cores, I know training on GPU is better but I only have access now on CPU.
Which parameters should I set using keras.models.Model.fit to utilize all ...
- 21
1
vote
0
answers
14
views
Are less training epochs better in the following scenario
So I have a scenario in which the training data is being generated in response to what the Neural Network backed actor is doing. In essence its giving feedback to the Neural Network based on all of ...
- 41
0
votes
0
answers
24
views
Generating unique points with an auto-encoder
I have been working on some research using a type of auto-encoder to generate new points with specific desirable properties. I trained my network and successfully generated some points, but when I ...
1
vote
1
answer
4k
views
1D target tensor expected, multi-target not supported
I am trying to train my model.
My model outputs a [4,2] tensor where 4 is the batch size and 2 because of binary classification.
After receiving the outputs I found the index of the maximum element ...
- 15
0
votes
1
answer
167
views
Aren't balanced data sets important in regression?
Why is it that the necessity for balanced data sets is (almost) always exclusively mentioned in the context of classification but not of regression?
- 203
1
vote
3
answers
393
views
Spliting Training Test and Validation for Image Dataset
I have 600 images in the training folder, 200 images in the validation folder, and 200 images in the test folder. Suppose if I fit the training data generator and validation data generator for some ...
- 36
0
votes
2
answers
1k
views
No target response variable in my testing data
I have two datasets which are the training and testing set. The training data has a target variable, but the testing set does not. What should I do to fix the issue with the testing set?
- 31
1
vote
1
answer
2k
views
How train - test split works for Graph Neural Networks
I have recently started studying GNN's. I have covered GCN and GraphSage so far. But I am confused regarding the process when testing occurs.
Now suppose in the graph above I am using the nodes as ...
- 113