Questions tagged [training]
Training is the part of machine learning whereby a model is "trained" on a define portion of a dataset to learn attributes and statistical features of the data. It's counterparts are called Testing and Validation. After training a model is tested and validated on another portion of the dataset.
680
questions
123
votes
2
answers
113k
views
Training an RNN with examples of different lengths in Keras
I am trying to get started learning about RNNs and I'm using Keras. I understand the basic premise of vanilla RNN and LSTM layers, but I'm having trouble understanding a certain technical point for ...
- 1,350
63
votes
6
answers
64k
views
Should a model be re-trained if new observations are available?
So, I have not been able to find any literature on this subject but it seems like something worth giving a thought:
What are the best practices in model training and optimization if new observations ...
- 1,783
55
votes
5
answers
30k
views
Is it always better to use the whole dataset to train the final model?
A common technique after training, validating and testing the Machine Learning model of preference is to use the complete dataset, including the testing subset, to train a final model to deploy it on, ...
- 3,920
55
votes
4
answers
38k
views
What is the advantage of keeping batch size a power of 2?
While training models in machine learning, why is it sometimes advantageous to keep the batch size to a power of 2? I thought it would be best to use a size that is the largest fit in your GPU memory /...
- 1,175
41
votes
8
answers
9k
views
What would I prefer - an over-fitted model or a less accurate model?
Let's say we have two models trained. And let's say we are looking for good accuracy.
The first has an accuracy of 100% on training set and 84% on test set. Clearly over-fitted.
The second has an ...
- 519
37
votes
5
answers
48k
views
In the context of Deep Learning, what is training warmup steps
I found the term "training warmup steps" in some of the papers. What exactly does this term mean? Has it got anything to do with "learning rate"? If so, how does it affect it?
- 1,059
35
votes
9
answers
13k
views
Why is it wrong to train and test a model on the same dataset?
What are the pitfalls of doing so and why is it a bad practice? Is it possible that the model starts to learn the images "by heart" instead of understanding the underlying logic?
- 461
21
votes
4
answers
76k
views
Train, test split of unbalanced dataset classification
I have a model that does binary classification.
My dataset is highly unbalanced, so I thought that I should balance it by undersampling before I train the model. So balance the dataset and then ...
- 413
21
votes
6
answers
25k
views
Tool to label images for classification
Can anyone recommend a tool to quickly label several hundred images as an input for classification?
I have ~500 microscopy images of cells. I would like to assign categories such as ...
- 401
15
votes
1
answer
15k
views
Is stratified sampling necessary (random forest, Python)?
I use Python to run a random forest model on my imbalanced dataset (the target variable was a binary class). When splitting the training and testing dataset, I struggled whether to used stratified ...
- 783
11
votes
2
answers
17k
views
Oversampling/Undersampling only train set only or both train and validation set
I am working on a dataset with class imbalance problem. Now, I know one needs to oversample or undersample only the train set and not the test set. But my issue is: whether to oversample the train set ...
- 711
10
votes
2
answers
10k
views
Train object detection without annotated data/bounding boxes
From what I can see most object detection NNs (Fast(er) R-CNN, YOLO etc) are trained on data including ...
- 203
10
votes
3
answers
28k
views
How to split train/test datasets having equal classes proportion
I would like to know how I can split in an equal number the following
Target
0 1586
1 318
in order to have the same proportion of 0 and 1 classes in a ...
- 155
9
votes
3
answers
751
views
What knowledge do I need in order to write a simple AI program to play a game?
I'm a B.Sc graduate. One of my courses was 'Introduction to Machine Learning', and I always wanted to do a personal project in this subject.
I recently heard about different AI training to play games ...
- 93
9
votes
3
answers
6k
views
Build a tool for manually classifying training data images
I have a large number of images that I need to classify for training a clustering algorithm, and I would like to do so offline (the data is proprietary). Basically, I'd like to build a desktop survey ...
- 278
9
votes
1
answer
8k
views
How to train data by batch from disk?
I am working on a convolutional neural network for image classification. The training dataset is too large to be loaded on my computer memory (4gb), on top of that I also need to try some augmentation ...
8
votes
4
answers
3k
views
Understanding how convolutional layers work
After working with a CNN using Keras and the Mnist dataset for the well-know hand written digit recognition problem, I came up with some questions about how the convolutional layer work. I can ...
8
votes
2
answers
10k
views
How to deal with large training data?
Currently, I use image files and transform them into a *.npy file(saved as a numpy array) as training data. At present this training data set is nearly 3GB.
Now I have more image files, so the ...
- 93
8
votes
1
answer
1k
views
CNN for phoneme recognition
I am currently studying this paper, in which CNN is applied for phoneme recognition using visual representation of log mel filter banks, and limited weight sharing scheme.
The visualisation of log ...
- 619
7
votes
3
answers
25k
views
Meaning of stratify parameter
I'm training a Neural Network and I'm trying to divide my data into training and testing sets. I have a lot of output classes and for some of them I have as little as 2 examples, so I would like to ...
7
votes
1
answer
4k
views
What are the effects of clipping the reward in stability?
I am looking for stabilizing my results of DQN, I found clipping is one technique to do it but I did not understand it completely!
1- what are the effects of clipping the reward, clipping the ...
- 1,824
7
votes
2
answers
3k
views
How to fix class imbalance in training sample?
I was very recently asked in a job interview about solutions to fix an imbalance of classes in the training dataset. Let's focus on a binary classification case.
I offered two solutions: oversampling ...
7
votes
1
answer
5k
views
Why is predicted rainfall by LSTM coming negative for some data points?
I have used supervised learning with LSTM network using tanh activation function and 0.1 dropout for time series prediction.my loss='mean_squared_error', optimizer='adam'. The predicted time series is ...
- 81
7
votes
3
answers
22k
views
What can be the cause of a sudden explosion in the loss when training a CNN (Deeplab)
I am training the following deeplab CNN:
https://github.com/tensorflow/models/tree/master/research/deeplab
During training I see the following loss:
The first 50k steps of the training the loss is ...
- 171
7
votes
1
answer
1k
views
How much text is enough to train a good embedding model?
I need to train a word2vec embedding model on Wikipedia articles using Gensim.
Eventually, I will use the entire Wikipedia for that
but for the moment, I'm doing some experimentation/optimization to ...
- 221
6
votes
4
answers
13k
views
Tool to Label Images for Supervised Classification
I have a couple thousand photos of whales taken from drones and I'm planning to build a simple binary classifier to run on these and future images to see if they contain a whale. I'd like to label ...
- 179
6
votes
1
answer
11k
views
Does increasing kernel size in a CNN result in higher accuracy on the training set?
In a convolutional neural network, does increasing the size of kernel always result in better training set accuracy? For example, if I use 5x5 kernels in a CNN instead of 3x3 ones, will it always ...
- 429
6
votes
1
answer
50k
views
Number of features of the model must match the input. Model n_features is `N` and input n_features is `X`.
I am new to data science and trying get some results. I'm applying Decision Tree Classifier. When my train and test datasets' size are not equal I get an error `...
- 133
6
votes
2
answers
6k
views
Cross validation when training neural network?
The standard setup when training a neural network seems to be to split the data into train and test sets, and keep running until the scores stop improving on the test set. Now, the problem: there is ...
- 3,142
6
votes
2
answers
406
views
Why real-world output of my classifier has similar label ratio to training data?
I trained a neural network on balanced dataset, and it has good accuracy ~85%. But in real world positives appear in about 10% of the cases or less. When I test network on set with real world ...
- 63
6
votes
1
answer
2k
views
Keras: do `sample weights` take part in the derivatives
According to Keras documentation, sample_weight can be used in order to give any sample in the training data a different importance in the loss.
I have googled ...
- 61
6
votes
2
answers
4k
views
Why an increasing validation loss and validation accuracy signifies overfitting?
When I train a neural network, I observe an increasing validation loss, while at the same time, the validation accuracy is also increased.
I have read explanations related to the phenomenon, and it ...
- 99
6
votes
1
answer
7k
views
Possible to use different learning rate for different neuron in Keras/Tensorflow?
The simplest example is to have faster/slower learning rates in the upper/lower layers of a network. I found this post on tensorflow.
Is there a similar trick in Keras?
Going one step further, can ...
- 1,350
6
votes
1
answer
2k
views
Smart data split (train/eval) for Object Detection
I am looking for a smart way of splitting object detection data (images with labelled objects inside them) while taking into account the distribution of the objects themselves and not just the images.
...
- 61
6
votes
2
answers
898
views
Why do most GAN (Generative Adversarial Network) implementations have symmetric discriminator and generator architectures?
For example, if the discriminator is a vanilla network of n layers, each with n(i) units, then, typically, the generator will also be a vanilla network of n layers, each with n(n-i) units (except the ...
- 191
6
votes
1
answer
374
views
Repeated k-fold Cross Validation for time series data?
I have a relative small sample size (330 with 45 features) + it's time series data.
I want to train my LightGBM regression model for best generalized RMSE score and want to use repeated CV. I use ...
- 161
5
votes
2
answers
2k
views
Do model training pipeline should run on dev, staging and production environment?
I know it's a best practice to ship our code from dev to staging to production by including different level tests and validations that will help to confidently deploy on the production environment.
...
- 151
5
votes
1
answer
48
views
On design of the training set: conceptual question
I am curious to know how training data should be constructed so that it scales to examples that are not a part of the training data. For example, the problem that I am facing right now is in the ...
- 521
5
votes
3
answers
125
views
Can you learn an algorithm from a trained model?
Are there any papers where an algorithm was entirely based on the results of a trained model? Let me explain.
Suppose you want to come up with an algorithm that sorts three numbers $a,b,c$. I can ...
- 153
5
votes
1
answer
7k
views
What is the difference between tensorflow saved_model.pb and frozen_inference_graph.pb?
I've re-trained a model (following this tutorial) from the google's object detection zoo (ssd_inception_v2_coco) on a WIDER Faces Dataset and it seems to work if I use ...
- 51
5
votes
1
answer
2k
views
Transformer masking during training or inference?
I'm working through Attention is All you Need, and I have a question about masking in the decoder. It's stated that masking is used to ensure the model doesn't attend to any tokens in the future (not ...
- 63
5
votes
2
answers
4k
views
Batching in Recurrent Neural Networks (RNNs) when there is only a single instance per time step?
I have scoured the internet and books, but everything seems to use num_steps and batch_size or similar terms interchangeably and ...
- 159
5
votes
1
answer
1k
views
Why could an overfitted CNN model have a higher validation accuracy?
I am currently training a CNN model by using cifar10 images (50000 for training, another 10000 for validation).
I plot training loss, validation loss and accuracy against training iteration:
I am ...
- 65
5
votes
1
answer
3k
views
Train loss vs validation loss
I have a few basic questions about tracking losses during training.
If I am using mini-batch training, should I validate after each
batch update or after I have seen the entire dataset?
What should ...
- 213
5
votes
2
answers
841
views
Type of images used to train a neural network
I am a newbie in neural network.
Saw this article Object detection with deep learning and OpenCV.
These three type of neural network are shortlisted in the article
Faster R-CNNs
You Only Look Once (...
- 183
5
votes
1
answer
775
views
How many epochs to run during hyperparameter search?
If I'm doing a hyperparameter search and comparing two different hyperparameters (but not number of epochs), is there some established rule of thumb for how many epochs to run? If I just compare ...
- 161
4
votes
3
answers
3k
views
Very Fast Training After First Epoch
I trained an InceptionV3 model using plant images. I used Keras library. When training was started, first epoch took 29s per step and then other steps took approximately 530ms per step. So that made ...
- 422
4
votes
2
answers
502
views
Training a model sample by sample
I'm training a Scikit model but it seems that in all examples, they call the fit method on the entire training set. What I want to do however is call it per sample (...
4
votes
3
answers
13k
views
What do x and y mean when working with data in the machine learning domain?
We need train & test data, so what do x and y mean?
Does it mean that it divides 15% to the 'x_train', 'x_val', 'y_train', 'y_val' ?
...
- 41
4
votes
2
answers
2k
views
Is it correct to join training and validation set before inferring on test-set?
I would like to know if is a correct procedure to join training-set and validation-set together, in order to train the model on ...
- 695