New answers tagged tensorflow
0
One of the major issue is the amount of records per categories. Definitely using data augmentation is always helpful to get good accuracy but much better approach is to use transfer learning techniques with data augmentation.
In your approach model is started overfitting because of that it's started having good training accuracy but bad validation accuracy. ...
0
I am assuming from the description inside main directory there are more directories of Apples, Bananas, Oranges, etc. and inside them you have .txt files containing information about the images.
import shutil
with open(file_path, 'r') as f:
img_names = f.readlines()
img_names = [img.strip() for img in img_names]
for i in range(len(img_names)): ...
0
First, you need to convert the dataframe in numpy array or tf.data dataset that the model understands. For this purpose, the tutorial provides you with a function:
# A utility method to create a tf.data dataset from a Pandas Dataframe
def df_to_dataset(dataframe, shuffle=True, batch_size=32):
dataframe = dataframe.copy()
labels = dataframe.pop('target')
...
0
I guess the error in the example is that, instead of numbers objects were passed to HParams. You should handle the learning rate just as any other float parameter and just pass it to the optimizer. Actually hard to tell what you want exactly without a code example.
0
X.shape here as I guess is something similar to the mnist data, (60000, 28, 28), means it doesn't have extra dimension or say 24bit-representation, i.e., some color-bytes.
As such, each x in X is having 2D shape, thus, X.shape[1:] -eq x.shape -eq (28, 28).
You have to explicitly reshape X to include the extra dimension needed for Conv2D layer.
As per the ...
0
The easiest way I found was replacing flow_from_directory command to flow_from_dataframe (for more information on this command see).
That way you can split the dataframe. You just have to make a dataframe with images paths and labels.
i = 1
df_metrics = pd.DataFrame()
kf = KFold(n_splits = 10, shuffle = True, random_state = None)
for train_index,...
1
Check out this answer:
https://stackoverflow.com/a/60230236/12642230
Alternative Solution:
Tensorflow provides a method pad_sequences() to do that:
https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/sequence/pad_sequences
The default value of padding is 'pre', you might wanna change that to 'post' to do what you want, along with ...
1
Well, there are several ways you can do that.
A quite powerful solution is to define a pred layer
class PredLayer(Layer):
"""
Layer object to calculate distance between query_embeddings and supposrt embeddings.
"""
def __init__(self, **kwargs):
super(PredLayer, self).__init__(**kwargs)
def euclidean_distance(self, inputs):
...
0
If possible, try to use just tensorflow functions (put the .numpy() part out, for example), and put a @tf.function decorator on top of train() function.
The role of @tf.function is that it transforms a whole function into a tensorflow op. The whole function will be execute an order of magnitude faster than a normal Python function.
0
You're just overfitting here. That's a fairly complex network for the simple MNIST data set. It's fairly easy to separate the MNIST classes, so even though your overfit network is starting to do worse on the validation set - it's getting less certain about the correct answers, believing the wrong ones more - the most-probable class is still almost always ...
0
Usually this is due to a learning rate that is too high, it passes over the Loss function minimum and starts overshooting. Of course I can't be sure that's the reason but this is my best guess.
Try to simplify your optimizer, use Adam() optimizer alone (without moving average) and set a fairly small learning rate, something like 0.001 or even 0.0001. Let's ...
0
According to the github readme:
One or more high-end NVIDIA GPUs, NVIDIA drivers, CUDA 10.0 toolkit
and cuDNN 7.5. To reproduce the results reported in the paper, you
need an NVIDIA GPU with at least 16 GB of DRAM.
0
There are some options to run stuff on the GPU without having the user install CUDA:
If the user has the nvidia driver installed, you could bundle the CUDA libraries with your application (along with any other indirect dependency, maybe cublas and stuff like that). This is what some deep learning libraries like PyTorch do.
You could use tensorflow.js, ...
0
As you said:
Is it possible to preprocess the data in batches and train a model for the small data set
Yes! In practice, training of Neural Networks always happens with batches. You never fit the whole dataset at once in the model, whatever medium sized dataset could crash any machine.
This is how it works: You extract a slice of your dataframe (the ...
0
In that tutorial, they created a new BahdanauAttention() class that is supposed to be inserted into the Decoder() object. Attention is something the Decoder uses, not the Encoder. The model as it is isn't complete. Add a Decoder and change its input shape to make it work.
However, let me conclude with some thoughts on this implementation:
I don't think you ...
3
Octave is a great language for prototyping and experimenting with ML algorithms, as it has built-in support for numerical linear algebra such as matrix and vector calculations. Octave is optimized for rapid calculations, which is very useful in Machine Learning. It is also quite easy to do matrix multiplications in Octave as Matrices are first-class objects ...
2
Sounds like you're just looking for EarlyStopping, which will stop training when validation loss does not improve for N epochs. It's the same as Iter in catboost.
1
The reason is tfx doesn't support Keras sequential model, but function API. After migrating to use function API, it works in fine in the trainer pipeline.
def get_model(show_summary=True):
#one-hot categorical features
num_A = 4,
num_B = 3,
num_C = 2,
num_D = 8,
num_E = 12,
num_F = 4,
num_G = 16,
num_H = 26
input_A = tf.keras.Input(shape=(num_A,), name="...
1
Is the answer already there?
"
The function can capture non-linear interactions between features. You can think of the lattice parameters as the height of poles set in the ground on a regular grid, and the resulting function is like cloth pulled tight against the four poles.
With features and 2 vertices along each dimension, a regular lattice will have ...
0
You might be underfitting and so have to (at least) train for a longer time (more epochs). Are you calculating your accuracy in the training set before calculating the for the testing set (as the case in many neural network implementation)? Is each training accuracy a mean from many training steps? If this is the case and you are not overfitting (too much), ...
0
If you train your model and then run model.evaluate N times (do not retrain the model) you should get the same answer each time PROVIDED your test data is the SAME each time. However if you train your model then run evaluate and do that combination N times results will vary due to the random weight initialization of the network.
0
It doesn't seem that it is about the size of the model. I am no SageMaker expert, but the error message suggests that the model was deployed, but that something went wrong when the health check was run.
This could be caused by many different things, but the most probable would be that there is a bug in the code. Please check the following:
Can the model be ...
0
This is the closer answer I can find.
# Input
inputs = Input(shape=(max_length,), name="Input")
# Embedding
embed = Embedding(input_dim=n_words+1,
output_dim=embedding_size,
input_length=max_length,
name="Embedding")(inputs)
# Bi-LSTM
encoder = Bidirectional(LSTM(units=hidden_state_encoder_size,
...
0
Try this, and tune alpha.
model = tf.keras.models.Sequential([
keras.layers.Flatten(input_shape=(28,28)),
keras.layers.Dense(128,activation=tf.keras.layers.LeakyReLU(alpha=0.3)),
keras.layers.Dense(10,activation=tf.nn.softmax)
])
1
You can use the ds.map() function to create dataset conatains only images or labels:
ds_images = ds.map(lambda d:d['images')
the original purpose of the map function is manipulating the data without converting to numpy, for example:
ds_images = ds.map(lambda d:d['images']/255)
hope I helped you.
0
Does this work for you? After the Bi-Directional LSTM layer, I sum the tensor over the batch axis and then divide by the batch size.
from tensorflow.keras import layers, Sequential
from tensorflow.keras.models import Model
from tensorflow.keras import backend as K
from tensorflow.keras.utils import plot_model
inputs = layers.Input(batch_shape=(4, 500), ...
0
First of all, keep this in mind:
After all, if it was easy to do this without any labels, then, what would be the point of needing the labels in the first place?
I can see two options:
Use a pre-trained image classifier to represent your images
As Vincent Young suggests, you can find pre-trained networks which have been trained on similar detection tasks....
1
Call model_name.summary()
This will return you the correct value for the total number of parameters.
2
from keras.applications.resnet50 import ResNet50
resnet_model = ResNet50(weights='imagenet')
#resnet_model.count_params()
resnet_model.summary()
Total params: 25,636,712
Trainable params: 25,583,592
Non-trainable params: 53,120
Check your code once to be sure that it is ResNet50
2
The advantages of training a deep learning model from scratch and of transfer learning are subjective. It depends a lot on the problem you are trying to solve, the time constraints, the availability of data and the computational resources you have.
Let's consider a scenario, you want to train a deep learning model for a task like sentiment classification ...
0
Alright, so... I got help from someone. He mentions changing LSTM to return_sequences.
# Input
inputs = Input(shape=(max_length,), name="Input")
# Embedding
embed = Embedding(input_dim=n_words+1,
output_dim=embedding_size,
input_length=max_length,
name="Embedding")(inputs)
# Bi-LSTM
encoder = ...
3
(Suggestions and edits will be appreciated)
let us discuss advantages of training a deep learning model from scratch:
Building and training NN from scratch is of a great use in the research field.
You will know your model to the most basics and can modify it in case needed as per the requirements.
It will be more efficient in terms of size and training ...
0
I think there might be a number of errors here:
There is no need to convert X to categorical, only your labels.
Hence, there is not need for this line: tf.keras.utils.to_categorical(X, num_classes=None, dtype="float32")
to_categorical is not inplace so you would need to reassign it to y. Change it to: y = tf.keras.utils.to_categorical(y)
Once you convert ...
0
To get around this issue, I would suggest doing training in batches. This solves the issue of not being able to load 100,000 images and also removes the need to figure out how to merge the models.
0
Overfitting is a result of the entire model. It is best to visualize every part of the model to understand how the combination of elements is overfitting.
Ideally, you should visualize all elements on the same interactive figure so how the combination is overfitting. If that is not possible or the interpretation is difficult, then many single element ...
0
This means that your input shape was too small by the time it reached the last Conv2D layer.
By removing the last Conv2D and MaxPool2D layer, this is how the model looks like:
You can see that the last output before the Flatten layer has height and width of 4, which is smaller than your kernel_size of 5 in the last Conv2D layer.
Heres what you can do:
...
0
It seems I was returning multiple features as labels. I had to modify the dense_1_step function to return a single feature...
def dense_1_step(batch):
# Shift features and labels one step relative to each other.
return batch[:-1], batch[-1:,1][0] # take second feature only
To make it the same as the output from the multivariate_data function.
0
The problem is inside the sampling functions. I had the same problem and found out the answer in the tutorial here.
my original code is:
def sampling(args):
z_mean, z_log_sigma = args
epsilon = K.random_normal(shape=z_mean.shape)
return z_mean + K.exp(z_log_sigma) * epsilon
with this sampling method, I got the same error with yours.
the ...
1
Can I ask you if you are using any form of dropout?
It happened to me before that because I did apply dropout to the training set, but not on the validation set, I would easily get higher accuracy in the validation
2
It seems weird.
Your validation has a higher score than your training. This literally means that your model performs better in unseen data than what it sees.
Typical underfitting is that you achieve the same in train than in test.
In my opinion, since you are not providing much information, you are not splitting right the data. It might be for a lot of ...
2
There are not only 2, but many implementations of BERT. Most are basically equivalent.
The implementations that you mentioned are:
The original code by Google, in Tensorflow. https://github.com/google-research/bert
Implementation by Huggingface, in Pytorch and Tensorflow, that reproduces the same results as the original implementation and uses the same ...
0
The big issue here is that your generator yields after each file is loaded. This means that your batch size is always the number of examples stored in each file and that the training examples in each batch are always the same.
You can absolutely shuffle the order of the files, but that's just shuffling the batch order around and not shuffling data examples ...
0
Some quick notes:
A mini-batch size of 4096 is rather large, try training with sizes like 16, 32, or even 1 to see what happens.
Your test accuracy seems to be around random since there are 9/10 classes to predict. Does your model predict randomly (i.e. every class gets a similar score) or is it just wrong often?
is it necessary to implement it from ...
0
On further investigation was able to narrow it down.
The loss value is not only calculated by the response from loss function, but also the various regularizers added in the intermediate layers.
And one of the FC layers in my model was indeed having a L2 regularization term as below:
model.add(layers.Dense(1024, activation='relu', kernel_regularizer=...
0
Figured it out. To train certain sub-networks first and re-use the trained weights to initialize other layers:
from keras.layers import Input, Dense
from keras.models import Model
main_input = Input(shape=(5, ))
## Model A: main_input -> A_output
layer_A1 = Dense(10, name='A1')(main_input)
layer_A2 = Dense(10, name='A2')(layer_A1)
layer_A3 = Dense(10, ...
0
First, If you calculate the mean along dim=1 the output shape should be [a, c].
If you want to mask the mean that's less then a threshold and set it to zero you can do
# generate data
torch.manual_seed(42)
a, b, c = 2, 3, 4
t = torch.normal(torch.ones(a, b, c))
print(t)
tensor([[[ 2.9269, 2.4873, 1.9007, -1.1055],
[ 1.6784, -0.2345, 0.9569, -0....
0
Adding more images in the training set is one way to increase accuracy. You can also do transfer learning, i.e. using layers trained by a larger dataset if you are classifying common things such as animals which have networks that have already been trained such as the ResNet50.
When you have enough images that the accuracy does not increase anymore, you can ...
Top 50 recent answers are included
