Questions tagged [training]
Training is the part of machine learning whereby a model is "trained" on a define portion of a dataset to learn attributes and statistical features of the data. It's counterparts are called Testing and Validation. After training a model is tested and validated on another portion of the dataset.
682
questions
1
vote
1
answer
169
views
Help training Back-propagation Neural Network With 150k training pairs
I am currently trying to train my backpropagation to classify 150k training pairs. Each training pair is a vector of 18 Bipolar numbers and it runs through 2 hidden layers with a final output of 1 ...
3
votes
1
answer
556
views
Understanding the training phase of the tutorial "Using Keras and Deep Deterministic Policy Gradient to play TORCS" tutorial
I am trying to understand the training phase of the tutorial Using Keras and Deep Deterministic Policy Gradient to play TORCS (mirror, code) by Ben Lau published on October 11, 2016.
The tutorial ...
- 5,650
8
votes
1
answer
1k
views
CNN for phoneme recognition
I am currently studying this paper, in which CNN is applied for phoneme recognition using visual representation of log mel filter banks, and limited weight sharing scheme.
The visualisation of log ...
- 619
0
votes
3
answers
565
views
Using training data generated with pure regular expressions - Can machine learning surpass the accuracy of your regular expression?
For text classification with machine learning - If your training data was generated purely with regular expressions, is it possible to train a machine learning model with this training data which will ...
1
vote
1
answer
575
views
GANs to augment training data
I have been reading about Generative Adversarial Networks (GANs) and was wondering if it would make sense to train a generator function only to use it for creating more training data.
In a scenario ...
- 113
1
vote
0
answers
60
views
Predict arguments of indicator function or the value the indicator function itself
I have an evenly-spaced timeseries $a_1, a_2, a_3,..., a_n$ and a function $f_k=\mathbf 1(a_k-a_{k-5})$, where $\mathbf 1$ is the unit step function.
I want to predict $f_k$ using $a_1, a_2, a_3,..., ...
- 111
2
votes
1
answer
6k
views
Predict customer action from previous buying history
I'm trying to predict what service a customer wants when he comes to our office from his previous transactions history. I have 7 years transactions data(3 crore txns) and good amount of customers are ...
- 163
1
vote
0
answers
359
views
Tips for retraining convolutional neural networks given a drastically different loss surface
For the image dataset I am working with, I need to use B&W version of images (otherwise, I would need to build a network to give false colors to a set of my images, since they have an overpowering ...
- 56
3
votes
0
answers
214
views
Multiple models vs. Single model for prediction
I am using the Darknet Convolutional Neural Networks to detect people (as in, humans) and furniture in a single image.
If I train the model twice, one for people, one for furniture. I seem to get ...
- 131
15
votes
1
answer
15k
views
Is stratified sampling necessary (random forest, Python)?
I use Python to run a random forest model on my imbalanced dataset (the target variable was a binary class). When splitting the training and testing dataset, I struggled whether to used stratified ...
- 783
1
vote
0
answers
64
views
Spark Deeplearning4j Training Problem
I am training a model in Spark using Dl4J library in Yarn-Cluster mode. When I train the model on 2 lakh data (approx 200MB) then the job succeeds but when I go to train the model with 3 lakh data (...
- 81
2
votes
5
answers
9k
views
The model performance vary between different train-test split?
I fit my dataset to the random forest classifier and found that the model performance would vary among different sets of train and test data split. As what I have observed, it would jump from 0.67 to ...
- 783
9
votes
3
answers
751
views
What knowledge do I need in order to write a simple AI program to play a game?
I'm a B.Sc graduate. One of my courses was 'Introduction to Machine Learning', and I always wanted to do a personal project in this subject.
I recently heard about different AI training to play games ...
- 93
6
votes
1
answer
7k
views
Possible to use different learning rate for different neuron in Keras/Tensorflow?
The simplest example is to have faster/slower learning rates in the upper/lower layers of a network. I found this post on tensorflow.
Is there a similar trick in Keras?
Going one step further, can ...
- 1,350
1
vote
6
answers
20k
views
Which train test split performs better: 50:50 or 60:40? [closed]
I have 10,000 customer data of a supermarket. And I want to split the data into training set and testing set. So, which train test split gives me a better accuracy: 50:50 or 60:40?
- 111
-1
votes
1
answer
63
views
What is the minimum or maximum amount of trained data is used to train a classifier? [closed]
what should be the min or max size of trained datasets should be used to feed a classifier ? can we use 1 GB or more data as trained data to feed a classifier for jvm related ML frameworks?
- 389
8
votes
2
answers
10k
views
How to deal with large training data?
Currently, I use image files and transform them into a *.npy file(saved as a numpy array) as training data. At present this training data set is nearly 3GB.
Now I have more image files, so the ...
- 93
1
vote
2
answers
238
views
Time complexity of function minimizers for neural networks
I am trying to train a neural network for recognizing handwritten letters from A to J . I have a training set of size 200000 . Each training set is a list of 784 ...
- 227
4
votes
4
answers
241
views
Extract 2 pieces of information from a string - what to use?
First of all, I am a complete newbie in regard to data science and I am not asking for the complete solution but some guidance as to what I should read up to achieve my task (what algorithms, ...
- 141
21
votes
6
answers
25k
views
Tool to label images for classification
Can anyone recommend a tool to quickly label several hundred images as an input for classification?
I have ~500 microscopy images of cells. I would like to assign categories such as ...
- 401
0
votes
1
answer
65
views
Picking training data
Suppose i want to have 80% training data and 20% testing data. How do i choose which 80% of the data to use for training? Should it be completely random? Like what if there is a class label with 2 ...
- 419
1
vote
1
answer
79
views
Query regarding neural network model
I used the Neural Network Toolbox in matlab to train my data. I used four training algorithms, Scaled Conjugate Gradient (SCG), Gradient Descent with momentum and adaptive learning back-propagation (...
- 1,161
9
votes
3
answers
6k
views
Build a tool for manually classifying training data images
I have a large number of images that I need to classify for training a clustering algorithm, and I would like to do so offline (the data is proprietary). Basically, I'd like to build a desktop survey ...
- 278
0
votes
2
answers
112
views
Does the phenomenon of over-fitting of data varies with training algorithms?
Suppose I have a dataset which I want to train using Neural network and SVM. Is it possible that with my dataset after training, the Neural network is overfit while the SVM is not? Like can a dataset ...
- 1,161
0
votes
1
answer
278
views
Deep learning - rule generation
I wanted to know if there is any methodology in Deep/Machine learning, where given a set of input/output values, it can derive rules for the same.
Lets say I generate training input and output by $y=...
- 177
63
votes
6
answers
65k
views
Should a model be re-trained if new observations are available?
So, I have not been able to find any literature on this subject but it seems like something worth giving a thought:
What are the best practices in model training and optimization if new observations ...
- 1,783
1
vote
0
answers
223
views
Generating a text training dataset from a grammar
I want to generate documents based on a grammar to build a custom training database. What are the tools and techniques to generate random texts based on a given grammar.
More specifically, I would ...
- 513
0
votes
1
answer
335
views
Please suggest some good ways to test my RNN
I have developed an RNN in matlab and now its time to test it. You can set your desired number of layers and nodes and it trains data in random chunks. I also included an annealing function for ...
- 1
1
vote
1
answer
3k
views
Training And Testing Error Curves caret package in r
I am Running the following models
Logistic regression
Decision Trees
SVM
Naive Bayes
Random Forest
On the same data set.
I am using Caret package in r.
Its My dream to plot Training error and ...
2
votes
3
answers
1k
views
How big should training data be?
I am solving a problem connected with medicine and from each patient I get about 100 features. It's a classification diseases problem, however measurements take a lot of time and also require money.
...
- 73
4
votes
1
answer
232
views
Using GPS signal, determine is this person driving a cab
New York City provides tens of gigs of data of taxi routes all over the city. What I'd like to do, is use this data (or some other method), to come up with an algorithm that can take a persons GPS ...
- 141
6
votes
2
answers
6k
views
Cross validation when training neural network?
The standard setup when training a neural network seems to be to split the data into train and test sets, and keep running until the scores stop improving on the test set. Now, the problem: there is ...
- 3,142