Skip to main content

Questions tagged [training]

Training is the part of machine learning whereby a model is "trained" on a define portion of a dataset to learn attributes and statistical features of the data. It's counterparts are called Testing and Validation. After training a model is tested and validated on another portion of the dataset.

Filter by
Sorted by
Tagged with
125 votes
2 answers
116k views

Training an RNN with examples of different lengths in Keras

I am trying to get started learning about RNNs and I'm using Keras. I understand the basic premise of vanilla RNN and LSTM layers, but I'm having trouble understanding a certain technical point for ...
Tac-Tics's user avatar
  • 1,370
63 votes
6 answers
66k views

Should a model be re-trained if new observations are available?

So, I have not been able to find any literature on this subject but it seems like something worth giving a thought: What are the best practices in model training and optimization if new observations ...
neural-nut's user avatar
  • 1,783
57 votes
5 answers
34k views

Is it always better to use the whole dataset to train the final model?

A common technique after training, validating and testing the Machine Learning model of preference is to use the complete dataset, including the testing subset, to train a final model to deploy it on, ...
pcko1's user avatar
  • 3,950
57 votes
4 answers
43k views

What is the advantage of keeping batch size a power of 2?

While training models in machine learning, why is it sometimes advantageous to keep the batch size to a power of 2? I thought it would be best to use a size that is the largest fit in your GPU memory /...
James Bond's user avatar
  • 1,195
43 votes
5 answers
58k views

In the context of Deep Learning, what is training warmup steps

I found the term "training warmup steps" in some of the papers. What exactly does this term mean? Has it got anything to do with "learning rate"? If so, how does it affect it?
Ashwin Geet D'Sa's user avatar
41 votes
8 answers
9k views

What would I prefer - an over-fitted model or a less accurate model?

Let's say we have two models trained. And let's say we are looking for good accuracy. The first has an accuracy of 100% on training set and 84% on test set. Clearly over-fitted. The second has an ...
EitanT's user avatar
  • 519
35 votes
9 answers
15k views

Why is it wrong to train and test a model on the same dataset?

What are the pitfalls of doing so and why is it a bad practice? Is it possible that the model starts to learn the images "by heart" instead of understanding the underlying logic?
karalis1's user avatar
  • 461
21 votes
4 answers
79k views

Train, test split of unbalanced dataset classification

I have a model that does binary classification. My dataset is highly unbalanced, so I thought that I should balance it by undersampling before I train the model. So balance the dataset and then ...
lads's user avatar
  • 413
21 votes
6 answers
25k views

Tool to label images for classification

Can anyone recommend a tool to quickly label several hundred images as an input for classification? I have ~500 microscopy images of cells. I would like to assign categories such as ...
jlarsch's user avatar
  • 401
15 votes
1 answer
15k views

Is stratified sampling necessary (random forest, Python)?

I use Python to run a random forest model on my imbalanced dataset (the target variable was a binary class). When splitting the training and testing dataset, I struggled whether to used stratified ...
LUSAQX's user avatar
  • 783
12 votes
2 answers
19k views

Oversampling/Undersampling only train set only or both train and validation set

I am working on a dataset with class imbalance problem. Now, I know one needs to oversample or undersample only the train set and not the test set. But my issue is: whether to oversample the train set ...
yamini goel's user avatar
10 votes
2 answers
11k views

Train object detection without annotated data/bounding boxes

From what I can see most object detection NNs (Fast(er) R-CNN, YOLO etc) are trained on data including ...
salient's user avatar
  • 203
10 votes
3 answers
34k views

How to split train/test datasets having equal classes proportion

I would like to know how I can split in an equal number the following Target 0 1586 1 318 in order to have the same proportion of 0 and 1 classes in a ...
user105599's user avatar
9 votes
3 answers
756 views

What knowledge do I need in order to write a simple AI program to play a game?

I'm a B.Sc graduate. One of my courses was 'Introduction to Machine Learning', and I always wanted to do a personal project in this subject. I recently heard about different AI training to play games ...
Niv Hoffman's user avatar
9 votes
3 answers
6k views

Build a tool for manually classifying training data images

I have a large number of images that I need to classify for training a clustering algorithm, and I would like to do so offline (the data is proprietary). Basically, I'd like to build a desktop survey ...
atkat12's user avatar
  • 278
9 votes
1 answer
8k views

How to train data by batch from disk?

I am working on a convolutional neural network for image classification. The training dataset is too large to be loaded on my computer memory (4gb), on top of that I also need to try some augmentation ...
Learning is a mess's user avatar
8 votes
4 answers
3k views

Understanding how convolutional layers work

After working with a CNN using Keras and the Mnist dataset for the well-know hand written digit recognition problem, I came up with some questions about how the convolutional layer work. I can ...
Karampistis Dimitrios's user avatar
8 votes
2 answers
10k views

How to deal with large training data?

Currently, I use image files and transform them into a *.npy file(saved as a numpy array) as training data. At present this training data set is nearly 3GB. Now I have more image files, so the ...
nsknsl's user avatar
  • 93
8 votes
3 answers
25k views

What can be the cause of a sudden explosion in the loss when training a CNN (Deeplab)

I am training the following deeplab CNN: https://github.com/tensorflow/models/tree/master/research/deeplab During training I see the following loss: The first 50k steps of the training the loss is ...
MuadDev's user avatar
  • 181
8 votes
1 answer
1k views

CNN for phoneme recognition

I am currently studying this paper, in which CNN is applied for phoneme recognition using visual representation of log mel filter banks, and limited weight sharing scheme. The visualisation of log ...
Carlton Banks's user avatar
7 votes
3 answers
26k views

Meaning of stratify parameter

I'm training a Neural Network and I'm trying to divide my data into training and testing sets. I have a lot of output classes and for some of them I have as little as 2 examples, so I would like to ...
Carlos A. Jimenez Holmquist's user avatar
7 votes
1 answer
4k views

What are the effects of clipping the reward in stability?

I am looking for stabilizing my results of DQN, I found clipping is one technique to do it but I did not understand it completely! 1- what are the effects of clipping the reward, clipping the ...
user10296606's user avatar
  • 1,844
7 votes
1 answer
6k views

Why is predicted rainfall by LSTM coming negative for some data points?

I have used supervised learning with LSTM network using tanh activation function and 0.1 dropout for time series prediction.my loss='mean_squared_error', optimizer='adam'. The predicted time series is ...
Roy's user avatar
  • 81
7 votes
2 answers
3k views

How to fix class imbalance in training sample?

I was very recently asked in a job interview about solutions to fix an imbalance of classes in the training dataset. Let's focus on a binary classification case. I offered two solutions: oversampling ...
Learning is a mess's user avatar
7 votes
1 answer
1k views

How much text is enough to train a good embedding model?

I need to train a word2vec embedding model on Wikipedia articles using Gensim. Eventually, I will use the entire Wikipedia for that but for the moment, I'm doing some experimentation/optimization to ...
Abdulrahman Bres's user avatar
6 votes
2 answers
2k views

Do model training pipeline should run on dev, staging and production environment?

I know it's a best practice to ship our code from dev to staging to production by including different level tests and validations that will help to confidently deploy on the production environment. ...
shaik moeed's user avatar
6 votes
4 answers
13k views

Tool to Label Images for Supervised Classification

I have a couple thousand photos of whales taken from drones and I'm planning to build a simple binary classifier to run on these and future images to see if they contain a whale. I'd like to label ...
clifgray's user avatar
  • 179
6 votes
2 answers
3k views

Is it correct to join training and validation set before inferring on test-set?

I would like to know if is a correct procedure to join training-set and validation-set together, in order to train the model on ...
Simone's user avatar
  • 715
6 votes
1 answer
12k views

Does increasing kernel size in a CNN result in higher accuracy on the training set?

In a convolutional neural network, does increasing the size of kernel always result in better training set accuracy? For example, if I use 5x5 kernels in a CNN instead of 3x3 ones, will it always ...
Saptarshi Roy's user avatar
6 votes
1 answer
50k views

Number of features of the model must match the input. Model n_features is `N` and input n_features is `X`.

I am new to data science and trying get some results. I'm applying Decision Tree Classifier. When my train and test datasets' size are not equal I get an error `...
Mutafaf's user avatar
  • 133
6 votes
2 answers
6k views

Cross validation when training neural network?

The standard setup when training a neural network seems to be to split the data into train and test sets, and keep running until the scores stop improving on the test set. Now, the problem: there is ...
Alex I's user avatar
  • 3,152
6 votes
2 answers
468 views

Why real-world output of my classifier has similar label ratio to training data?

I trained a neural network on balanced dataset, and it has good accuracy ~85%. But in real world positives appear in about 10% of the cases or less. When I test network on set with real world ...
Bien's user avatar
  • 63
6 votes
1 answer
2k views

Keras: do `sample weights` take part in the derivatives

According to Keras documentation, sample_weight can be used in order to give any sample in the training data a different importance in the loss. I have googled ...
user92151's user avatar
6 votes
2 answers
5k views

Why an increasing validation loss and validation accuracy signifies overfitting?

When I train a neural network, I observe an increasing validation loss, while at the same time, the validation accuracy is also increased. I have read explanations related to the phenomenon, and it ...
glorian's user avatar
  • 99
6 votes
1 answer
8k views

Possible to use different learning rate for different neuron in Keras/Tensorflow?

The simplest example is to have faster/slower learning rates in the upper/lower layers of a network. I found this post on tensorflow. Is there a similar trick in Keras? Going one step further, can ...
horaceT's user avatar
  • 1,370
6 votes
1 answer
2k views

Smart data split (train/eval) for Object Detection

I am looking for a smart way of splitting object detection data (images with labelled objects inside them) while taking into account the distribution of the objects themselves and not just the images. ...
CarlosUziel's user avatar
6 votes
2 answers
1k views

Why do most GAN (Generative Adversarial Network) implementations have symmetric discriminator and generator architectures?

For example, if the discriminator is a vanilla network of n layers, each with n(i) units, then, typically, the generator will also be a vanilla network of n layers, each with n(n-i) units (except the ...
Antoine Savine's user avatar
6 votes
1 answer
473 views

Repeated k-fold Cross Validation for time series data?

I have a relative small sample size (330 with 45 features) + it's time series data. I want to train my LightGBM regression model for best generalized RMSE score and want to use repeated CV. I use ...
Vega's user avatar
  • 161
5 votes
3 answers
4k views

Very Fast Training After First Epoch

I trained an InceptionV3 model using plant images. I used Keras library. When training was started, first epoch took 29s per step and then other steps took approximately 530ms per step. So that made ...
tkarahan's user avatar
  • 442
5 votes
3 answers
10k views

Alternatives with better GPU than Google Colab Pro

I am currently running/training MAchine learning models that are very GPU expensive, Google Colab Pro is not giving me enough GPU/RAM Is there any alternatives with better GPU and more RAM than ...
The Dan's user avatar
  • 193
5 votes
1 answer
50 views

On design of the training set: conceptual question

I am curious to know how training data should be constructed so that it scales to examples that are not a part of the training data. For example, the problem that I am facing right now is in the ...
Sm1's user avatar
  • 541
5 votes
2 answers
5k views

Resampling for imbalaced datasets: should testing set also be resampled?

Apologies for what is probably a basic question but I have not been able to find a definitive answer either in the literature or in the Internet. When dealing with an imbalanced dataset one possible ...
Jose Manuel Albornoz's user avatar
5 votes
3 answers
128 views

Can you learn an algorithm from a trained model?

Are there any papers where an algorithm was entirely based on the results of a trained model? Let me explain. Suppose you want to come up with an algorithm that sorts three numbers $a,b,c$. I can ...
AspiringMat's user avatar
5 votes
1 answer
7k views

What is the difference between tensorflow saved_model.pb and frozen_inference_graph.pb?

I've re-trained a model (following this tutorial) from the google's object detection zoo (ssd_inception_v2_coco) on a WIDER Faces Dataset and it seems to work if I use ...
vi-kun's user avatar
  • 51
5 votes
1 answer
3k views

Transformer masking during training or inference?

I'm working through Attention is All you Need, and I have a question about masking in the decoder. It's stated that masking is used to ensure the model doesn't attend to any tokens in the future (not ...
David Rein's user avatar
5 votes
2 answers
4k views

Batching in Recurrent Neural Networks (RNNs) when there is only a single instance per time step?

I have scoured the internet and books, but everything seems to use num_steps and batch_size or similar terms interchangeably and ...
ehiller's user avatar
  • 159
5 votes
1 answer
1k views

Why could an overfitted CNN model have a higher validation accuracy?

I am currently training a CNN model by using cifar10 images (50000 for training, another 10000 for validation). I plot training loss, validation loss and accuracy against training iteration: I am ...
Lei Xun's user avatar
  • 65
5 votes
1 answer
3k views

Train loss vs validation loss

I have a few basic questions about tracking losses during training. If I am using mini-batch training, should I validate after each batch update or after I have seen the entire dataset? What should ...
pg2455's user avatar
  • 213
5 votes
2 answers
856 views

Type of images used to train a neural network

I am a newbie in neural network. Saw this article Object detection with deep learning and OpenCV. These three type of neural network are shortlisted in the article Faster R-CNNs You Only Look Once (...
Santhosh's user avatar
  • 183
5 votes
1 answer
883 views

How many epochs to run during hyperparameter search?

If I'm doing a hyperparameter search and comparing two different hyperparameters (but not number of epochs), is there some established rule of thumb for how many epochs to run? If I just compare ...
Wes's user avatar
  • 161

1
2 3 4 5
14