New answers tagged tensorflow
0
Installed a dualboot a week ago on a similar setup. I didn't use Pytorch or TF yet, but I didn't ran into much struggle for CUDA and CudNN installation. Here is what I've done and the few problems I've run into :
Downloaded Ubuntu 20.04 LTS, created a USB bootable with Rufus.
Created a 30 GB partition on my disk (only 10GB are required, but why not), ...
0
Your post is a duplicate : Here someone asked the same question.
Have a look at the answers, you can also try this link that gives a detailled explanation about how to reshape inputs for LSTM layers.
0
Vanishing gradients can occur because of adding so many layers to network. You can think neural networks as composite functions. During learning process gradients of loss function with respect to weights calculated according to chain rule, and because of large number of layers gradients can be so small by multiplications. It causes insufficient updates of ...
0
Very interesting statement you got there, i was not aware of this 'optimal loss value'.
the only way i could explain it is with the gradient computation : gradient is computed using :
$Grad = loss*lr$
So to me, changing the loss range is the same as changing the learning rate of your optimizer, it could be a good experiment to increase your lr instead of ...
1
d_model is the dimensionality of the representations used as input to the multi-head attention, which is the same as the dimensionality of the output. In the case of normal transformers, d_model is the same size as the embedding size (i.e. 512). This naming convention comes from the original Transformer paper.
depth is d_model divided by the number of ...
0
What you want to do is weird, and i'm not sure i understand it correctly, but i assume you could use the answer to this post.
4
Your model is probably overfitting. You could try following things (or mixture of these):
Remove layers or reduce the number of neurons
Use dropout technique
Use regularization (e.g L1,L2)
Use Data Augmentation
copied from: https://www.kdnuggets.com/2019/12/5-techniques-prevent-overfitting-neural-networks.html
9
This means your model is overfitting the train set. Try simplifying the model by reducing the number of neurons in each layer. That will reduce your train accuracy but may allow your validation accuracy to increase.
You know that you are not overfitting when both train and validation accuracy hover around the same value.
1
It seems that you have cuda 10 installed, and not version 11 which is what TensorFlow is looking for. It has rather specific requirements to get working.
Tensorflow 2.4.1 requires cuda 11 and cudnn 8 (see GPU table of requirements here). I would suggest checking also, your nvidia driver version, 450.x or higher is required.
This is the TensorFlow official ...
0
if you look into the code you can figure what's exactly happening.
Take tf.keras.layers.experimental.preprocessing.RandomRotation for example
def call(self, inputs, training=True):
...
def random_flipped_inputs():
flipped_outputs = inputs
if self.horizontal:
flipped_outputs = image_ops.random_flip_left_right(flipped_outputs,
...
0
That is a common case on image and audio processing, you need to find a way in which dimensions stay the same, such as normalizing per channel.
If you have a 1D vector of features, taking mean and variance of all variables will end up normalizing it in a way, it works in Computer Vision like a charm. It is also a way to reduce the space cost of your ...
0
It is the learning rate that you set on the optimizer for your problem. It requires a fair bit of experimentation to figure out what learning rate works best for your problem. It is a more of an art than a science, though there exists heuristics on an optimal range of values to select from.
For most problems, the default learning rate should be a good ...
3
First, note that they are just adding 1 to the size of the vocabulary, not to the token IDs themselves, so the predictions are not affected.
Then, why adding 1 ?
Because Tokenizer.word_index is a python dictionary that contains token keys (string) and token ID values (integer), and where the first token ID is 1 (not zero) and where the token IDs are assigned ...
1
You may try cycle GANs.
In normal Generative Adversarial Networks (GANs), a generative network -the generator- is trained to generate images with an auxiliary network, the discriminator. The discriminator learns to tell apart data generated by the generator (i.e. fake data) from real data, while the generator learns to generate data that fools the ...
0
It seems like your model hasn't learnt much and may almost always be predicting the mean of the target distribution. Please check if the mean of the actual target variable resides in the 1260-90 range.
This may happen if your features are not correlated to the target.
Also, by looking at your graph, the MAE is certainly larger than 5%.
You want to collect ...
0
Given that the covariance matrix has to be positive definite, the cholesky decomposition is a good way to solve this problem.
So the output of the network will be the mean vector mu and the upper triangular part of the cholesky matrix (denoted T here). The diagonal of this matrix must be positive elements (the diagonal of the covariance matrix are standard ...
1
The reason the tensor takes up so much memory is because by default the tensor will store the values with the type torch.float32. This data type will use 4kb for each value in the tensor (check using .element_size()), which will give a total of ~48GB after multiplying with the number of zero values in your tensor (4 * 2000 * 2000 * 3200 = 47.68GB). What you ...
0
In a regular fully connected layer (Dense), the computation is done using the following Matrix operation : $$R = A*W + B$$
With all matrixes being vectors (if batch size = 1), exept $W$, which has size(inputsize, outputsize).
for a 3D input, the way TF computes the output is simply by applying this formula only to the last dimension, considering all other ...
1
One is a straight forward use of API while the other one gives you much better control on training. If you are experimenting, it is good to go with a custom training loop so you can control the various things that happen to your model's training.
But, if you want to just train a model and not experiment with it much, save time and go for the straight forward ...
1
Figured it out. Need to normalize the image I'm going to predict to $[-1,1]$ instead of what-I-assume-is-the-standard $[0,1]$. It seems this is model-dependent. The project I'm working off of uses transfer learning and imports this model.
It's not immediately clear to me though just how this model needs inputs normalized to $[-1,1]$.
0
...Because the Keras documentation does not specify the keys for the class_weights..
You may get an idea with these two parameters,
labels: Either "inferred" (labels are generated from the directory structure), or a list/tuple of integer labels of the same size as the number of image files found in the directory. Labels should be sorted according ...
1
Glad you found where it went wrong! However, it is really possible for something like that to happen. There is no such thing as "best algorithm", so the performance of a method partly depends on what your dataset looks like. Or sometimes your feature engineering method just allows the data to cheat on you, say, you mistakenly leaked some data, or ...
2
[I've added this answer as I think others miss the main theoretical gist.]
Firstly, NCE and Negative Sampling (NS) serve different purposes:
NS is a generic trick used to train a classifier if you only have training samples from one `positive' class (e.g. labelled $y\!=\!1$);
NCE is a method to learn parameters $\theta$ of a model $p_m(x;\theta)$ of a true ...
0
A couple of things that jump out immediately as being a little odd:
You seem to have two labels "ball_count" and "run_per_ball" but only 1 output of the network.
You are using an MSE loss, which is typically used for regression problems. This seems to be a sensible choice for outputs like ball_count or run_per_ball. However, accuracy, is ...
0
Use a standard pretrained network like ResNet, Effnet or even VGG16 for feature extraction and put a regression layer in the end.
Now, talking about your network. FIrst of all, the image size is not square, make it! Don't use even valued kernels (3 and 5 are mostly used). Number of filters increase with depth, yours don't follow a pattern.
Lastly, 100 images ...
0
Everything seems correct to my non-expert eyes in your code, exept lr in the code you shared (0.00001) seems low for a SGD, but as you mentionned, you tried different rates, so this probably is not the issue there.
I'm not familiar with elu or selu activation functions and usually use relu in all my layers, yet i can't say if that is the problem.
May you ...
1
Using model.fit() will always train on the whole dataset for n epochs. The batch_size argument denotes how many samples are used to calculate the gradient and updates the parameters. If you want to train the model on just a batch of data (i.e. a subset of your total dataset) simply pass the subset of data to model.fit().
0
Thanks to everyone who gave answer and comments. It was indeed caused by my data.
Prior to this I had the same preprocessing pipeline for both models, which would be your "usual" NLP preprocessing steps (non-alphanumerical removal, lowercasing, stemming, and stop word removal). I had a hunch that both stemming and stop word removal would cause the ...
1
This is overfitting, and it suggests that your images in each class are very similar to images across other classes.
Since your images across classes seem very similar, 800 per class is actually not a lot of data to train on. It's likely your model is struggling to discriminate the dev data into the correct classes based on what little it can learn., and ...
1
Yes, this could be possible if your dev/test data comes from the same domain as the training data, in which case word2vec will encounter fewer OOV tokens that mess up the loss.
This could also mean that the benefits of BERT - subword tokenization to handle OOV characters in generalized domains - are lost. If your vocabulary size is small, your word2vec model ...
0
Different operations on different elements don't prevent differentiation in any way.
Lets, say we call your above Loss function:
$$\mathcal L=L_1(\mathbf w) + L_2(\mathbf w)$$,
where $\mathbf w$ represents the weights of your model, $L_1$ and $L_2$ are the two loss functions you have defined using the different outputs of your model. The key point is that ...
0
I finally was able to solve my issue. It's weird but even though the custom multicategory layer did not have params it contained its own mapping for the data. To extend the model and to examine the effect of layer depths I created a new model by adding the multicategory layer from the existing model. Once I did this training accuracy matched AutoKeras.
0
The answer can be found by just printing
sample_model.summary()
giving
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense (Dense) (None, 32) 160 ...
0
In order to stack list of tf tensors you could use the tf function stack hence the name
for ur case,
tf.stack([some_function(x) for x in something],axis=0)
or you could stack them in numpy and then convert the array to a tensor, to do so using numpy np.stack inputting a list and axis
Top 50 recent answers are included
