# Tag Info

### The cross-entropy error function in neural networks

One way to interpret cross-entropy is to see it as a (minus) log-likelihood for the data $y_i'$, under a model $y_i$. Namely, suppose that you have some fixed model (a.k.a. "hypothesis"), which ...
• 2,101
Accepted

### How to disable GPU with TensorFlow?

I've seen some suggestions elsewhere, but they are old and do not apply very well to newer TF versions. What worked for me was this: ...
• 1,060
Accepted

### What does from_logits=True do in SparseCategoricalcrossEntropy loss function?

The from_logits=True attribute inform the loss function that the output values generated by the model are not normalized, a.k.a. logits. In other words, the softmax ...
• 774
Accepted

### Multi GPU in Keras

From the Keras FAQs, below is copy-pasted code to enable 'data parallelism'. I.e. having each of your GPUs process a different subset of your data independently. ...
• 636

### Why does Keras need TensorFlow as backend?

This makes more sense when understood in its historical context. These were the chronological events: April 2009 Theano 0.1 is released. It would dominate the deep ...
• 16.7k

### What is the relationship between the accuracy and the loss in deep learning?

There is no relationship between these two metrics. Loss can be seen as a distance between the true values of the problem and the values predicted by the model. Greater the loss is, more huge is the ...
• 932
Accepted

### Merging two different models in Keras

I figured out the answer to my question and here is the code that builds on the above answer. ...
• 1,013
Accepted

• 1,336
Accepted

### Intuitive explanation of Noise Contrastive Estimation (NCE) loss?

Taken from this post:https://stats.stackexchange.com/a/245452/154812 The issue There are some issues with learning the word vectors using an "standard" neural network. In this way, the word vectors ...
• 524

### The cross-entropy error function in neural networks

The first logloss formula you are using is for multiclass log loss, where the $i$ subscript enumerates the different classes in an example. The formula assumes that a single $y_i'$ in each example is ...
• 27.7k
Accepted

### In CNN, why do we increase the number of filters in deeper Convolution layers for complex images?

For this you need to understand what filters actually do. Every layer of filters is there to capture patterns. For example, the first layer of filters captures patterns like edges, corners, dots etc. ...
• 837
Accepted

### Difference between RMSProp with momentum and Adam Optimizers

(My answer is based mostly on Adam: A Method for Stochastic Optimization (the original Adam paper) and on the implementation of rmsprop with momentum in Tensorflow (which is operator() of struct ...
• 634
Accepted

### How to use LeakyRelu as activation function in sequence DNN in keras?When it perfoms better than Relu?

You can use the LeakyRelu layer, as in the python class, instead of just specifying the string name like in your example. It works similarly to a normal layer. Import the LeakyReLU and instantiate a ...
• 14.2k

### How to define a custom performance metric in Keras?

You have to use Keras backend functions. Unfortunately they do not support the &-operator, so that you have to build a workaround: We generate matrices of the ...
• 381

### What is weight and bias in deep learning?

Mathematically speaking, Imagine you are a model (No! not that kind, figure 8 ones). Bias is simply how biased you are. If you are a Nigerian, and you are asked "Which nationality have the most ...
• 321

### Keras vs. tf.keras

From Keras repo.: Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano. And Before installing Keras, please install one of ...
• 8,767

### What is the meaning of "The number of units in the LSTM cell"?

Most LSTM/RNN diagrams just show the hidden cells but never the units of those cells. Hence, the confusion. Each hidden layer has hidden cells, as many as the number of time steps. And further, each ...
• 381
Accepted

### How to calculate the output shape of conv2d_transpose?

Here is the correct formula for computing the size of the output with tf.layers.conv2d_transpose(): ...
• 316
Accepted

### What is one hot encoding in tensorflow?

Suppose you have a categorical feature in your dataset (e.g. color). And your samples can be either red, yellow or blue. In order to pass this argument to a ML algorithm, you first need to encode it ...
• 7,658
Accepted

### How to deal with string labels in multi-class classification with keras?

Sklearn's LabelEncoder module finds all classes and assigns each a numeric id starting from 0. This means that whatever your class representations are in the ...
• 27.7k
Accepted

### Using TensorFlow with Intel GPU

At this moment, the answer is no. Tensorflow uses CUDA which means only NVIDIA GPUs are supported. For OpenCL support, you can track the progress here. BTW, Intel/AMD CPUs are supported. The default ...
• 4,156

### Custom loss function with additional parameter in Keras

You can write a function that returns another function, as is done here on GitHub ...
• 9,178
Accepted

### Tensorflow neural network TypeError: Fetch argument has invalid type

The problem lay in using the name 'cost' on two occasions, the problem was solved by changing this: ...
• 453
Accepted

### Why TensorFlow can't fit simple linear model if I am minimizing absolute mean error instead of the mean squared error?

I tried this and got same result. It is because the gradient of .abs is harder for a simple optimiser to follow to the minima, unlike squared difference where ...
• 27.7k

### Neural Network for Multiple Output Regression

What you are describing is a normal multidimensional linear regression. This type of problem is normally addressed with a feed-forward network, either MLP or any other architecture that suits the ...
• 16.7k

### Keyword/phrase extraction from Text using Deep Learning libraries

This is an open area of research and it certainly depends on the way you frame the problem. If you're talking about multi-document summarization then the problem is slightly different than if you were ...
Accepted

### Keyword/phrase extraction from Text using Deep Learning libraries

The Google Research Blog should be helpful in the context of TensorFlow. In the above article, there is a reference to the Annotated English Gigaword dataset which is routinely used for text ...
Accepted

### What is the meaning of "The number of units in the LSTM cell"?

As the helpful comments in that function say, The definition of cell in this package differs from the definition used in the literature. In the literature, cell refers to an object with a ...
• 355

### Using TensorFlow with Intel GPU

You might want to check out https://github.com/benoitsteiner/tensorflow-opencl/ which is a fork of Tensorflow with OpenCL support. If your OS is supported by the fork and you are able to properly ...
• 161