New answers tagged

0 votes

Why is online learning 'much faster' than full-batch? - Neural Networks

It depends on the definition of full-batch and online learning. If full-batch is defined as n > 1 and online learning is defined as n = 1, then full-batch will be slower than online. Additional ...
user avatar
0 votes

What does this statement relative to neural network weight initialization mean?

If all the weights contribute equally, there is no way to single out some specific weights to penalise during backpropagation. Therefore the change in weights is just a trivial global rescaling.
user avatar
  • 196
0 votes
Accepted

What does this statement relative to neural network weight initialization mean?

Consider the following image of a simple neural network. Note that the network uses a linear activation function and that there are no bias terms (this makes the intuition easier). Each path from the ...
user avatar
0 votes

how to handle soft weight constraints in neural network

Maybe instead of mapping nn weights Wi to your problem, map Constant * Softmax(Wi) instead for each layer?
user avatar
  • 176
5 votes
Accepted

MLOps for beginner

a. For a beginner I would suggest the fullstackdeeplearning course, it's a modern overview of tools and best practices for ML in production. As you can see below, there are a lot of moving pieces. b. ...
user avatar
  • 196
5 votes

MLOps for beginner

You can do live learning but most models don't require it, because many businesses don't need to learn directly from new input. Nevertheless, you can apply an automated task every time range (day, ...
user avatar
0 votes

evaluation of gradient for the subset of parameters using backpropagation

If you are using Tensorflow, you can use the tf.GradientTape function: tf.GradientTape( persistent=False, watch_accessed_variables=True ) For example: ...
user avatar
0 votes

Newly discovered learning rule

I don't know how this algorithm performs, even with comments, because many functions are connected and use frequent loops, and they include also the output values. I have to track every step, and it ...
user avatar
0 votes

Odd error when training neural network with Keras - Error occurred when finalizing GeneratorDataset iterator

If it stops always after 36 epochs, it could be due to the data in input. You should check if the generated data has always the same dimension and same format, without any outlier (like an empty ...
user avatar
2 votes

What what will happen if all the layers of a MLP or any DL architecture are set as same in the beginning?

There is no correlation between underfitting/overfitting with the initialization of the weights. The problem with initialization of the weight is related to 2 main things: Back propagation: if you ...
user avatar
0 votes

Small difference in metrics in KERAS for the same model

I found explanation here: https://github.com/tensorflow/tensorflow/issues/29964 https://stackoverflow.com/questions/59118430/keras-model-evaluate-on-training-and-val-set-differ-from-the-acc-and-val-...
user avatar
2 votes

Scalar predictor - is it better to have a lot of training data that is less precise? Or fewer training data that is more precise?

super-interesting question! My approach to the problem would be not to do any preprocessing on the data. This is, feed all the experiments to the network with the target being the 0/1 variable ...
user avatar
  • 489
1 vote

Scalar predictor - is it better to have a lot of training data that is less precise? Or fewer training data that is more precise?

I would say that it's better to have more data, since noise in the data is reduced by the optimization algorithm when optimizing (and therefore cause no problem in the optimization phase), and you can ...
user avatar
0 votes

change parameterization to eliminate weight constraints in neural networks

For other people asking this question: In keras / tensorflow I observed way better results using custom constraints instead of a parameterization. When using a parameterization, the weights stayed ...
user avatar
1 vote

Building machine learning models whilst penalizing them for complexity

Decision trees have many options for reducing overfitting. Examples include: Maximum depth Minimum node size Pruning Another method to reduce overfitting is adding a cost function penalty. L1 or L2 ...
user avatar
3 votes
Accepted

How does gradient descent avoid local minimums?

It does not. Gradient descent is not immune to local minima in non-convex function optimization. Nevertheless, the noise introduced by stochastic gradient descent (SGD) helps escaping local minima. ...
user avatar
  • 15.4k
1 vote

All classification models except neural network giving 100% accuracy

It could be due to a lack of initialization of your neurons. Did you initialize them randomly? For instance: ...
user avatar
3 votes
Accepted

How do researchers actually code novel architectures and layers?

In this particular case, I don't know how are they implementing these complex layers, but in Keras/TensorFlow you can define your own layers by inheriting from ...
user avatar
  • 489
1 vote

Why ReLU is better than the other activation functions

I think normalization issues are minor issues in numerical computations in general and machine learning in particular if you do not encounter significant numerical overflow or underflow. The expected ...
user avatar
  • 111
1 vote
Accepted

Loss function to prevent estimator bias

Thank you @Nikos M. for your suggestions. I was about to use your post-applied factor but then gave it another try. And found what caused this. It was that the final layer was using a ...
user avatar
0 votes

How to deal with different amounts of data every day?

One common option is to aggregate the embeddings of all the headlines. For example, you can compute the average of the embeddings and use the resulting 300-dim vector as input of the model.
user avatar
  • 489

Top 50 recent answers are included