66
This suggests that all the training examples have a fixed sequence length, namely timesteps.
That is not quite correct, since that dimension can be None, i.e. variable length. Within a single batch, you must have the same number of timesteps (this is typically where you see 0-padding and masking). But between batches there is no such restriction. During ...
20
When new observations are available, there are three ways to retrain your model:
Online: each time a new observation is available, you use this single data point to further train your model (e.g. load your current model and further train it by doing backpropagation with that single observation). With this method, your model learns in a sequential manner and ...
20
There are a couple of nuances here.
Complexity question very important - ocams razor
CV - is this trully the case 84%/83% (test it for train+test with CV)
Given this, personal opinion: Second one.
Better to catch general patterns. You already know that first model failed on that because of the train and test difference. 1% says nothing.
19
This is a problem of alignment of the virtual processors (VP) onto the physical processors (PP) of the GPU. Since the number of PP is often a power of 2, using a number of VP different from a power of 2 leads to poor performance.
You can see the mapping of the VP onto the PP as a pile of slices of size the number of PP.
Say you've got 16 PP.
You can map 16 ...
16
Once a model is trained and you get new data which can be used for training, you can load the previous model and train onto it. For example, you can save your model as a .pickle file and load it and train further onto it when new data is available. Do note that for the model to predict correctly, the new training data should have a similar distribution as ...
15
Best way is to collect more data, if you can.
Sampling should always be done on train dataset. If you are using python, scikit-learn has some really cool packages to help you with this. Random sampling is a very bad option for splitting. Try stratified sampling. This splits your class proportionally between training and test set.
Run oversampling, ...
14
Interesting question. I personally haven't seen that for products going into production, but understand the logic.
Theoretically, the more data your deployed model has seen, the better is should generalise. So if you trained the model on the full set of data you have available, it should generalise better than a model which only saw for example train/val ...
14
Once you have obtained optimal hyperparamters for your model, after training and cross validating etc., in theory it is ok to train the model on the entire dataset to deploy to production. This will, in theory, generalise better.
HOWEVER, you can no longer make statistical / performance claims on test data since you no longer have a test dataset.
If you ...
13
If the number of values belonging to each class are unbalanced, using stratified sampling is a good thing. You are basically asking the model to take the training and test set such that the class proportion is same as of the whole dataset, which is the right thing to do. If your classes are balanced then a shuffle (no stratification needed here) can ...
11
It depends mostly on the problem context. If predictive performance is all you care about, and you believe the test set to be representative of future unseen data, then the first model is better. (This might be the case for, say, health predictions.)
There are a number of things that would change this decision.
Interpretability / explainability. This is ...
8
I do something similar with keras and GPU training, where i also have only a small memory amount available. The idea would be split the numpy files into smaller ones, let's say 64 samples per file and then load each file and call train_on_batch on those images. You can use keras' train_on_batch function to achieve this:
train_on_batch
train_on_batch(self, ...
8
There are multiple ways to approach solving game playing problems. Some games can be solved by search algorithms for example. This works well for card and board games up to some level of complexity. For instance, IBM's Deep Blue was essentially a fast heuristic-driven search for optimal moves.
However, probably the most generic machine learning algorithm ...
8
As you are working on image classification and would also like to implement some data augmentation, you can combine the two AND load the batches directly from a folder using the mighty 'ImageDataGenerator` class.
Have a look at the execellent documentation!
I won't copy and paste the example from that link, but I can outline the steps that you go through:
...
7
It highly depends on the type of game and the information about the state of the game that is available to your AI.
Some of the most famous game playing AIs from last few years are based on deep reinforcement learning (e.g. Playing Atari with Deep Reinforcement Learning), which is normal reinforcement learning (e.g. Q-learning) with a deep neural network as ...
7
Is there any model in machine learning that does not have parameters?
Yes. k-nearest neighbors is parameterless (there is only a single hyper-parameter $k$).
If such parameterless models exist, what are their purpose then? Isn't it the whole point of training to tune a model's parameters?
Exactly: such models require no training at all. k-NN in ...
7
@kbrose seems to have a better solution
I suppose the obvious thing to do would be to find the max length of any sequence in the training set and zero pad it.
This is usually a good solution. Maybe try max length of sequence + 100. Use whatever works best for your application.
But then does that mean I can't make predictions at test time with input ...
7
Definitions, so we are on the same page:
Training set: the data points used to train the model.
Validation set: the data points to keep checking the performance of your model in order to know when to stop training.
Testing set: the data points used to check the performance once training is finished.
May training and validation sets overlap?
They should ...
6
A point that needs to be emphasized about statistical machine learning is that there are no guarantees. When you estimate performance using a held-out set, that is just an estimate. Estimates can be wrong.
This takes some getting used to, but it's something you're going to have to get comfortable with. When you say "What if the performance actually ...
6
You are supposed to pass numpy arrays and not lists as arguments to the DecisionTree, since your input was a list it gets trained as 70 features (1D list) and your test had list of 30 elements and the classifier sees it as 30 features.
Nonetheless, you need to reshape your input numpy array and pass it as a matrix
meaning: X_train.values.reshape(-1, 1) ...
5
I just hacked together a very basic helper in python
it requires that all images are stored in a pyton list allImages.
import matplotlib.pyplot as plt
category=[]
plt.ion()
for i,image in enumerate(allImages):
plt.imshow(image)
plt.pause(0.05)
category.append(raw_input('category: '))
5
I couldn't say what the authors refer to by
best by cross-validation
but I'll mention a simple and general procedure that's out there:
You are correct that analyzing one estimate of the generalization performance using one training and one test set is quite simplistic. Cross-validation can help us understand how this performance varies across datasets, ...
5
Yes, there are models that do this. This link points to one of the first papers I believe. The main idea is called weakly supervised object detection.
The paper essentially makes three modifications.
They treat the typical hidden fully connected layer as a convolutional layer. This works because convolutional layers can be thought of as convolving the same ...
5
If you are specifically looking to outline the whales, seems like FastAnnotationTool could work:
https://github.com/christopher5106/FastAnnotationTool
Other options here:
https://en.wikipedia.org/wiki/List_of_Manual_Image_Annotation_Tools
When had to annotate many images for a project, I built a fairly simple MATLAB gui that displayed images. I cycled ...
5
First and foremost, you need to reformat your data into what's called a balanced panel structure. For each day in your training data, each user should have a record for that day associated with an indicator variable for whether or not they visited. If every record in your training data corresponds to a visit, you're not giving those classifiers much to work ...
5
You problem is very common and many data scientists are struggling with there kind of issues.
In this blog post, the author explain very nicely what to do.
Those are the main notes:
1. Can You Collect More Data?
2. Try Changing Your Performance Metric:
Accuracy is not the metric to use when working with an imbalanced dataset. We have seen that it is ...
5
What can be the reason of such behavior?
A classifier only tries to capture the features-label relationship as accurate as it can, it does not learn nor does it guarantee that the ratio of predicted labels to be close to the true ratio. However, if sampled classes (even balanced) are good representatives of true class regions and classifier finds good ...
5
One of the basic assumptions governing Machine Learning is that samples from the training set must follow the same underlying distribution as samples from the test set (and so must any other sample you want fed to your model)!
This is why, usually, we randomly partition the same dataset into training and test sets.
This is actually one of the main reasons ...
5
In the general case this is impossible because standard learning algorithms require all the instances at once in order to compute the model. They usually work by making calculations based on the set of instances, then only the results of these calculations are stored as a model (for instance a linear regression classifier stores only the final weights which ...
5
The first has an accuracy of 100% on training set and 84% on test set. Clearly over-fitted.
Maybe not. It's true that 100% training accuracy is usually a strong indicator of overfitting, but it's also true that an overfit model should perform worse on the test set than a model that isn't overfit. So if you're seeing these numbers, something unusual is going ...
4
I think you are correct. The line should be
loss_batch, logits_batch = sess.run([loss, logits], feed_dict={X: X_batch, Y:Y_batch})
Only top voted, non community-wiki answers of a minimum length are eligible
