2

In self-attention, it is not the decoder attending the encoder, but the layer attends itself, i.e., the queries and values are the same. In practice, this is usually done in the multi-head setup. You can view that as every head focusing on collecting different kinds of information from the hidden states. In multi-headed attention with $H$ heads, you first ...


1

You are right, this code can't work. Adding d = n_hidden at the end of the loop should fix the problem. However, this not how you should work with TensorFlow. Fully-connected layers are a very routine thing and by implementing them manually you only risk introducing a bug. You should use Dense layer from Keras API and for the output layer as well.


1

Both learning rate $\eta$ and step size $\Delta w$ are linked to gradient descent. In the most simple case they are linked by : $$ \Delta w = w(t+1)-w(t) = -\eta \frac{\partial E(w)}{\partial w}$$ Where $t$ is the epoch and $E$ the error function. In that simple case, they only differ by $ - \frac{\partial E(w)}{\partial w} $, which sometimes lead to use ...


1

I do not know in which version of Tensorflow it was introduced, but at least in TF 2.1, there is tf.config.experimental_run_functions_eagerly(True) available. It makes all @tf.function-decorated functions run in eager mode anyway, until this is reset by calling with argument False again. For details see Tensorflow experimental_run_functions_eagerly ...


1

Tensorboard is an interactive visualization tool that reads log files and helps you track experiment metrics. For it to work with Keras's model.fit() method you need to add Tensorboard callback to model.fit() method. log_dir="logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S") tensorboard_callback = tf.keras.callbacks.TensorBoard( ...


1

You need to label data in order to train classifier. Labelled data are at the basis of all supervised ML. XML files are just one of the many format you can use to store information, people usually use them since they can be easily read by many tools.


1

Keras provides you high level api or can say wrapper written on top of multiple backends. These back ends have the core implementation of DNN. List of Keras supported backends are: Tensorflow Theano CNTK **Source: Keras documentation for supported backends Keras hides a bit complexity of DNN implementation, but again restrict your freedom. In case if ...


1

You cannot run the universal sentence encoder in reverse. There is no practical way to take an arbitrary embedding vector and get a sentence. My suggestion would instead be to find the sentence in your data with the embedding closest to your center. Euclidean distance works well, specially if you used K-means or another euclidean method to create your ...


1

In general in this situation you should be training to just the specific problem you are trying to solve - which from the sounds of it is just whether edges of any kind on the shape are blurry or cleanly printed. Unless you specifically need the shape labelled as a square triangle etc you should drop it as a label as will just add unnecessary complexity and ...


1

Accuracy is a statistic that you can compute on a dataset if you know the true labels. For a single image, the accuracy is either 0% or 100% based on if you get it right or wrong. In the newer versions of Keras, the predict method returns the probabilities of the classes, what you want to print is (if I guess correctly), the probability score for the best ...


1

It is better to formulate this problem in terms of classification, not regression. For example, you can apply sign transformation on your labels (e.g. $y = numpy.sign(y)$) before training the model, then fit the model with classification loss (like binary cross-entropy), and then you can use accuracy for measuring the performance. Using accuracy with ...


1

Just do tf.zeros_like(s) instead of using the shape explicitly.


1

Provided that you are in the same scope, will remember not only the learning rate but the current state of all tensor, hyper parameters, gradients and so on. In fact you can call fit many times instead of setting epochs and will work mostly the same.


Only top voted, non community-wiki answers of a minimum length are eligible