The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [training]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
14
votes
1answer
6k views

What is the advantage of keeping batch size a power of 2?

While training models in machine learning, why is it sometimes advantageous to keep the batch size to a power of 2? I thought it would be best to use a size that is the largest fit in your GPU memory /...
3
votes
1answer
822 views

Evaluating Logistic Regression Model in Tensorflow

Following this tutorial, I have a doubt about the evaluation part in: ...
1
vote
0answers
91 views

Inconsistent validation accuracy? is that expected?

I am currently training a pattern classification network, and seem to get very inconsistent result. The network seem to overfit, but the validation accuracy is very inconsistent. I am currently ...
1
vote
1answer
162 views

Prime model with mock data, good idea or not?

I am currently working on text classifier with some pretty unique characteristics. The data is composed of about 2K categories but 98% of the data lives in just one of those 2K categories. However, ...
2
votes
1answer
306 views

Training accuracy with SGD

So I am computing the training accuracy of my CNN on a batch training data and it almost always gives 100% accuracy. But I found from somewhere that in order to get the training accuracy, it is tested ...
0
votes
2answers
145 views

A Text Sections Classifier

I have a database of text files and I would like to classify each section(i.e references, abstract, etc...) in the text. For example: Abstract Indexes: 100-300 References Indexes: 9000-10000 ...
5
votes
2answers
3k views

Train object detection without annotated data/bounding boxes

From what I can see most object detection NNs (Fast(er) R-CNN, YOLO etc) are trained on data including ...
1
vote
0answers
759 views

Tensorflow Training Time Estimates

When I build models and before I train them, I have the habit of estimating the training time to make sure it is reasonable. This usually comes in the form of my naive estimate, training hours = ...
1
vote
1answer
126 views

Help training Back-propagation Neural Network With 150k training pairs

I am currently trying to train my backpropagation to classify 150k training pairs. Each training pair is a vector of 18 Bipolar numbers and it runs through 2 hidden layers with a final output of 1 ...
3
votes
1answer
540 views

Understanding the training phase of the tutorial “Using Keras and Deep Deterministic Policy Gradient to play TORCS” tutorial

I am trying to understand the training phase of the tutorial Using Keras and Deep Deterministic Policy Gradient to play TORCS (mirror, code) by Ben Lau published on October 11, 2016. The tutorial ...
7
votes
1answer
1k views

CNN for phoneme recognition

I am currently studying this paper, in which CNN is applied for phoneme recognition using visual representation of log mel filter banks, and limited weight sharing scheme. The visualisation of log ...
0
votes
3answers
317 views

Using training data generated with pure regular expressions - Can machine learning surpass the accuracy of your regular expression?

For text classification with machine learning - If your training data was generated purely with regular expressions, is it possible to train a machine learning model with this training data which will ...
1
vote
1answer
522 views

GANs to augment training data

I have been reading about Generative Adversarial Networks (GANs) and was wondering if it would make sense to train a generator function only to use it for creating more training data. In a scenario ...
1
vote
0answers
53 views

Predict arguments of indicator function or the value the indicator function itself

I have an evenly-spaced timeseries $a_1, a_2, a_3,..., a_n$ and a function $f_k=\mathbf 1(a_k-a_{k-5})$, where $\mathbf 1$ is the unit step function. I want to predict $f_k$ using $a_1, a_2, a_3,..., ...
2
votes
1answer
5k views

Predict customer action from previous buying history

I'm trying to predict what service a customer wants when he comes to our office from his previous transactions history. I have 7 years transactions data(3 crore txns) and good amount of customers are ...
1
vote
0answers
340 views

Tips for retraining convolutional neural networks given a drastically different loss surface

For the image dataset I am working with, I need to use B&W version of images (otherwise, I would need to build a network to give false colors to a set of my images, since they have an overpowering ...
0
votes
0answers
111 views

What is the best way to generate labelled data for an existing classifier?

I have an existing classification model which uses labelled data to do categorization of short texts. I am getting new data everyday and want to improve the classifier by making use of this data. So, ...
2
votes
0answers
118 views

Multiple models vs. Single model for prediction

I am using the Darknet Convolutional Neural Networks to detect people (as in, humans) and furniture in a single image. If I train the model twice, one for people, one for furniture. I seem to get ...
14
votes
1answer
12k views

Is stratified sampling necessary (random forest, Python)?

I use Python to run a random forest model on my imbalanced dataset (the target variable was a binary class). When splitting the training and testing dataset, I struggled whether to used stratified ...
1
vote
0answers
54 views

Spark Deeplearning4j Training Problem

I am training a model in Spark using Dl4J library in Yarn-Cluster mode. When I train the model on 2 lakh data (approx 200MB) then the job succeeds but when I go to train the model with 3 lakh data (...
1
vote
2answers
4k views

The model performance vary between different train-test split?

I fit my dataset to the random forest classifier and found that the model performance would vary among different sets of train and test data split. As what I have observed, it would jump from 0.67 to ...
8
votes
3answers
657 views

What knowledge do I need in order to write a simple AI program to play a game?

I'm a B.Sc graduate. One of my courses was 'Introduction to Machine Learning', and I always wanted to do a personal project in this subject. I recently heard about different AI training to play games ...
4
votes
1answer
5k views

Possible to use different learning rate for different neuron in Keras/Tensorflow?

The simplest example is to have faster/slower learning rates in the upper/lower layers of a network. I found this post on tensorflow. Is there a similar trick in Keras? Going one step further, can ...
0
votes
6answers
7k views

Which train test split performs better: 50:50 or 60:40?

I have 10000 customer data of a supermarket. And I want to split the data into training set and testing set. So,which train test split gives me a better accuracy?
-1
votes
1answer
56 views

What is the minimum or maximum amount of trained data is used to train a classifier? [closed]

what should be the min or max size of trained datasets should be used to feed a classifier ? can we use 1 GB or more data as trained data to feed a classifier for jvm related ML frameworks?
6
votes
2answers
8k views

How to deal with large training data?

Currently, I use image files and transform them into a *.npy file(saved as a numpy array) as training data. At present this training data set is nearly 3GB. Now I have more image files, so the ...
1
vote
2answers
166 views

Time complexity of function minimizers for neural networks

I am trying to train a neural network for recognizing handwritten letters from A to J . I have a training set of size 200000 . Each training set is a list of 784 ...
4
votes
4answers
178 views

Extract 2 pieces of information from a string - what to use?

First of all, I am a complete newbie in regard to data science and I am not asking for the complete solution but some guidance as to what I should read up to achieve my task (what algorithms, ...
7
votes
4answers
8k views

tool to label images for classification

Can anyone recommend a tool to quickly label several hundred images as an input for classification? I have ~500 microscopy images of cells. I want to assign categories such as 'healthy', 'dead', 'sick'...
0
votes
1answer
58 views

Picking training data

Suppose i want to have 80% training data and 20% testing data. How do i choose which 80% of the data to use for training? Should it be completely random? Like what if there is a class label with 2 ...
-2
votes
2answers
72 views

Training and Testing dataset

I am fitting a glm model on my data set. But in my case I don't have any competing models to decide upon, i.e., I just have only one model and have to estimate the coefficients of the model. In such a ...
1
vote
1answer
47 views

Query regarding neural network model

I used the Neural Network Toolbox in matlab to train my data. I used four training algorithms, Scaled Conjugate Gradient (SCG), Gradient Descent with momentum and adaptive learning back-propagation (...
7
votes
4answers
3k views

Build a tool for manually classifying training data images

I have a large number of images that I need to classify for training a clustering algorithm, and I would like to do so offline (the data is proprietary). Basically, I'd like to build a desktop survey ...
0
votes
2answers
85 views

Does the phenomenon of over-fitting of data varies with training algorithms?

Suppose I have a dataset which I want to train using Neural network and SVM. Is it possible that with my dataset after training, the Neural network is overfit while the SVM is not? Like can a dataset ...
0
votes
1answer
195 views

Deep learning - rule generation

I wanted to know if there is any methodology in Deep/Machine learning, where given a set of input/output values, it can derive rules for the same. Lets say I generate training input and output by $y=...
26
votes
3answers
23k views

Should a model be re-trained if new observations are available?

So, I have not been able to find any literature on this subject but it seems like something worth giving a thought: What are the best practices in model training and optimization if new observations ...
1
vote
0answers
184 views

Generating a text training dataset from a grammar

I want to generate documents based on a grammar to build a custom training database. What are the tools and techniques to generate random texts based on a given grammar. More specifically, I would ...
0
votes
1answer
266 views

Please suggest some good ways to test my RNN

I have developed an RNN in matlab and now its time to test it. You can set your desired number of layers and nodes and it trains data in random chunks. I also included an annealing function for ...
1
vote
1answer
2k views

Training And Testing Error Curves caret package in r

I am Running the following models Logistic regression Decision Trees SVM Naive Bayes Random Forest On the same data set. I am using Caret package in r. Its My dream to plot Training error and ...
2
votes
3answers
915 views

How big should training data be?

I am solving a problem connected with medicine and from each patient I get about 100 features. It's a classification diseases problem, however measurements take a lot of time and also require money. ...
4
votes
1answer
217 views

Using GPS signal, determine is this person driving a cab

New York City provides tens of gigs of data of taxi routes all over the city. What I'd like to do, is use this data (or some other method), to come up with an algorithm that can take a persons GPS ...
5
votes
2answers
4k views

Cross validation when training neural network?

The standard setup when training a neural network seems to be to split the data into train and test sets, and keep running until the scores stop improving on the test set. Now, the problem: there is ...