New answers tagged deep-learning
0
You should load images in batches. 480K of 224*224 sized images is too large for any machine let alone a 64GB one.
You can use image_generators from keras or tf.keras
https://www.tensorflow.org/tutorials/load_data/images#load_using_keraspreprocessing
https://keras.io/preprocessing/image/
1
Yes, testing data should follow the same preprocessing as the training data. Otherwise, testing data will have nothing comparable with what the algorithm learned, leading to (very) bad performances.
note: In Sklearn, the Pipeline class helps you to respect the fundamentals of ML modeling like data leakage and applying the same transformations to train and ...
0
One option is to switch to a deep learning framework that supports distributed training.
0
You could plot a graph of update versus iteration and analyze the variation of each update as the number of iteration increases. Like in here, where they are comparing the variance of the standard gradient descent algorithm versus its stochastic version.
0
If you are going to use the ECG time series as the input for the LSTM-network you will need to reshape your data
data = data.reshape(signal length,time step pr sample, ECG channel)
So if your data is 36000 samples long and you only use 1 channel you will do like this:
data = data.reshape(36000,1,1)
Then you can make a simple LSTM model like this:
model =...
0
It sounds like you want to do this: https://en.wikipedia.org/wiki/Object_detection. Without more information on your specific problem, there's not much more I can say, but suggest you start there.
0
The less data you have, the less complex your model can be. Otherwise you will overfit your data. There is not really a good way for me to judge what model is appropriate for you without knowing a lot about your dataset, but I doubt you will get anything sensible from 200 data points with a deep learning model. Try some simpler models like bag-of-words and ...
0
You have two options here that I can see:
1) quick and very very dirty - rebalance your data so that negatives are favored in the training data. Then your model will prefer to predict negative rather than positive, and false positives will be suppressed. To see this, consider what happens if all training data were negative. Tune the data balance until you ...
0
Your second hypothesis is on the right track. Try comparing the information content of the training set with the information content of the network parameters. Of course most images are compressible, but they don't compress down to a single floating-point number, which is how network parameters are usually encoded.
0
All neural networks can increase expressiveness and representational capacity by stacking layers. Each later layer can learn to non-linearly weigh the earlier layers. These non-linearities allow any function to be approximated. In the case of Recurrent Neural Network (RNN), it is functions over time. Stacked RNNs have increased abilities to learn functions ...
0
There is an example of classification in this repository:
DogVsCat
0
Because your two questions are phrased a bit differently, I'm going to answer the one in the title.
Is Neural Network Architecture independent of Data?
Generally speaking, yes.
One simple reason why you might have to change the architecture is if your new task has a different number of classes. If so you must change the last layer of your network.
...
0
SHORT ANSWER: Bayesian cost/benefit calculations directly tie "usefulness" to the evaluation of a model with metrics. Therefore, they are the only metrics (and there are an infinite number of them) which are actually useful.
For classification, use a Bayesian prior estimate for each class prevalence (relative class balance/imbalance) to convert a ...
0
RELU can only solve part of the gradient vanishing problem of RNN
because the gradient vanishing problem is not only caused by activation function.
see above function, the hidden state derivative will depend on both activation and Wrec, if Wrec's max eigen value < 1, the long term dependency's gradient will be vanished.
check the chain multiplication ...
0
By having seen the results on the held-out test set, learning something and applying that to the model, you have essentially added some bias to your model. So if we are being strict, yes... You cheated a little bit ;)
Without having a solid (meaning unbiased) validation metric, you cannot objectively compare the two results. You used prior information to ...
0
Input of Recurrent cells (LSTM but also GRU and basic RNN cells) follows this pattern:
( number of observations , lenght of input sequence , number of variables )
Assuming your lenght of input sequence is 3, and only one variable, you can go with:
LSTM(32, input_shape=(3, 1))
As you can see, when you declare an LSTM() layer you don't need to specify the ...
1
Gradient Descent minimizes the summation of costs for all data points in the training set. The weights in the network are universal (not specific to any class), and through gradient descent converge in such away that for each loss function all training data are minimized.
There are also multiple gradient descent algorithms. What you are describing here is ...
0
I realize this question was raised some time back but came across this and thought will share what I have done
It is better to use predict_classes function from the keras model rather than predict_generator - I have run into issues while using this with the time it takes to complete. However, the input data to this function will have to be an array which ...
0
I get your problem, the point is that its correct what the model does, but you have to build a look-up table for its answer. Your ground-truth, looks somethink like that [0,0,0,0,1], a one-hot vector for example. You, the human know what this code stands for, for example cats. just like that you have to build an numpy array, listing the word-embeddings in ...
1
So, this is just half answer, I write it here to be able to format the text clearly. You are facing troubles because you are trying to do something that you shouldn't, which is applying gradient to indices instead of embeddings. When using embeddings (all kinds, not only BERT), before feeding them to a model, sentences must be represented with embedding ...
2
Typically, resumes are not images. Almost all resumes are Microsoft Word or pdf. Given those document formats, a deep learning parser should be sequence-based (e.g., Recurrent Neural Network (RNN) or Long Short Term Memory (LSTM)).
To apply Deep Learning, you'll need many thousands of examples with each section labeled. There is HR-XML (Human Resources - ...
0
You have 1760 samples and you are using a pre-trained model with 12 epochs which has 20,815,148 trainable params. So, do not use the F-35 fighter to kill a mosquito. You better use a simpler model like simpler Neural Networks or even Logistic Regression.
0
None of the methods you described may classify a dataset alone whereas both can be used to transform your data into another domain in an unsupervised fashion.
PCA projects your data onto n-orthogonal components. A trained encoder (first component of the autoencoder) can project your data onto a latent space.
Both of those representations can be used in ...
1
3x3 conv, 256, /2
stands for:
3x3 Kernel
256 filters
a stride of 2 halving the spatial dimensions
The latter is explained on page 3 where the authors state
(ii) if the feature map size is halved, the number of filters is doubled so as to preserve the time complexity per layer. We perform downsampling directly by
convolutional layers that have a ...
0
Here is the full code
import numpy as np
import matplotlib.pyplot as plt
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.datasets import mnist
(X_train, Y_train), (X_test, Y_test) = mnist.load_data()
X_train = X_train / 255
X_test = X_test / 255
train_pixels=X_train.shape[1] * X_train.shape[2]
...
0
You are trying to input 60000 training images with size 28 by 28 into a dense neural network. This will work since a dense neural network can only work with one dimensional data, with each input neuron of the network representing a pixel in the image. You therefore have to reshape your data first from (n_samples, height, width) to (n_samples, n_pixels) using ...
1
In addition to the ncasas' answer, which is good in my opinion, I'd like to point out that ReLU is computationally inexpensive, in contrast to sigmoid activation functions. They require only an if / then comparison, while e.g. the logistic function requires exponentiation, addition, and division. This practical consideration makes ReLU's attractive, ...
0
Both Algorithm are quite similar, Only difference comes while iterating.In Gradient descent ,we consider all the points in calculating loss and derivative, while in Stochastic gradient descent, we use single point in loss function and its derivative randomly.
Checkout these two articles, both are inter-related and well explained.I hope it will help.
1....
0
Take a look at the python cv2 module. It has functions that should enable you to remove the watermark. If you have a separate image of the watermark and it is always in the same coordinate location in each image you should be able to subtract it from the images
1
Yes, at least you can identify what pixels' are contributing most in the prediction.
Tool like Layerwise Relevance Propagation, used for Explainable AI, serves the similar purpose and evaluate the values(weights) during back propagation and evaluate what pixels are contributing most.
Many opensource implementation are available and on similar track, ...
6
From what I've seen on Github while looking for open source projects is that people usually do both.
You can have a section where one loads the models and runs the inference, and another section where you let the user train the models from scratch using your code.
I recommend doing this since some people do not want to retrain the model, especially if it'...
0
I'm no expert myself, but recently (i.e. this is true to 2019), I've heard (at a meetup from an expert) that Node2Vec is the SOTA.
Here's a link to a post on Medium explaining it - basically, Node2Vec generates random walks on the graph (with hyper-parameters relating to walk length, etc), and embeds nodes in walks the same way that Word2Vec embeds words ...
0
Maybe it is caused by not correctly load pre-trained weights. Check if you use model.load_weights with by_name=True
1
It doesn't have to be the same function and usually is not.
The point of the validation set is to measure how well our model is actually doing. This is only useful if we measure it on a metric that has some value to us.
For instance, in classification, no one is concerned about which model achieves the least cross-entropy, rather which one achieves the ...
0
One approach would be t-distributed Stochastic Neighbor Embedding (t-sne) to learn the low-dimensional embedding.
The perplexity hyperparameter would have been tuned to get the outcome you desire. And since the method is probabilistic, it might have to be run multiple times to yield the most useful embedding.
The data in the low-dimensional embedding ...
0
Pandas is not memory efficient for data loading.
It appears you are using Keras / TensorFlow package. It would be better to use that package's data loading for csv which will better manage memory.
0
For synonyms I would directly use WordNet.
[added] For contextually similar words the traditional approach is to extract a context vector for every target word:
for every occurrence of a target word extract the words within a -/+ N window (e.g. N=5).
for every target word aggregate all its context words in a single context vector over the whole vocabulary.
...
0
F1 is based on hard classification; if the probability scores are hovering near the threshold, then the classifications may be flopping a lot, leading to unstable F1 scores.
A low F1 score is not too surprising in the presence of such imbalance; the default cutoff of 0.5 will often lead to high recall but low precision.
0
In simple terms, the learning rate affects how big a step you update your parameters to move towards the minimum point in your loss function. If your step is too big, you might miss the minimum point on your loss function. If your step is too small, it might take too long to converge to the minimum point.
Ways to deal with the above problems are the use of ...
0
The DQN uses experience replay to break correlations between sequential experiences. It is viewed that for every state, the next state is going to be affected by the current action, therefore, taking experiences sequentially would result in instabilities due to internal correlations between experiences. An experience consists of a state, an action, a reward ...
0
While Anomaly Detection is typically trained unsupervised (as mentioned in other answers), it is very beneficial to have a labeled dataset for validation and testing.
I would recommend labeling each time-period with an anomaly. If there are different kinds of anomalies (and these are known), then set up some classes of anomalies use those while labeling. ...
0
One option, which I have discovered, is back-translation via the Unsupervised Data Augmentation repository made public by Google Research. This is based on this paper.
0
Refer to this: https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv1D
The parameter filter determines the output channel.
0
There's a few options I would try:
Firstly, remove the class weight. See if it helps you get higher than 50% on a balanced dataset.
Secondly, either oversample Label 0 or undersample Label 1 instead of using the class weight.
Thirdly, try using focal loss as the loss function so your gradient updates are more focussed on the examples you are getting wrong....
0
Dropout helps improving performance of a machine learning model for the following reasons:
Making Network Simpler: It makes the network simpler hence, prevents over fitting.
Better than Using a Single Simple Network: It is better than manually re-designing a simpler network because once you have designed a particular architecture, you cannot change it until ...
2
A prior distribution expresses your assumptions about the model without observing any data. E.g. when doing linear regression, you a priori assume that the slope is close to zero. Now you start measureing data points and it turns out that the slope should probably be close to one, so you compromise and pick a value somewhat inbetween. If your belief in your ...
0
Gradient Descent need not always converge at global minimum.
It all depends on following conditions;
The function must be convex function.
What is convex function?
If the line segment between any two points on the graph of the function lies above or on the graph then it is convex function.
example is given below:
Less Learning rate(alpha), which means ...
0
In my experience with the natively LR, tinyface dataset, dlib's resnetv1 model failed to extract embeddings for a number of faces from images with high gaussian blur. However, FaceNet was able to vectorize those same poor-quality LR images.
0
RNN/LSTM is designed for series (data has time step) like data(E. g. a sentence ) which has dependency between different parts of the data. In English, some words in a sentence have a dependency on previous words.
To carry the dependency information and ignore the non-important information until the end of the sentence RNN/LSTM was introduced.
If you use ...
1
Because average of encoded pitches is not the encoded class.
Look at this example:
LabelEncoder can turn [dog,cat,dog,mouse,cat] into [1,2,1,3,2], but then the imposed ordinality means that the average of dog and mouse is cat. Still there are algorithms like decision trees and random forests that can work with categorical variables just fine and ...
Top 50 recent answers are included
